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SECTION  1 


INTRODUCTION 


This  report  documents  an  analysis  of  the  needs  of  depot-level  maintenance 
technicians  for  interactive  information  aids  for  job  performance,  Sophis tica ted 
aids  will  be  made  possible  by  the  introduction  of  the  Automated  Technical  Order 
System  (ATOS)  for  electronic  documentation  to  Air  Logistics  Centers  (ALCs).  The 
analysis  was  performed  under  contract  to  Battelle  by  Honeywell's  Systems  and 
Research  Center  (Contract  No.  DAAG2 9- 81-D-0100) . 

1.1  OBJECTIVES 


The  objective  of  the  study  was  to  explore  the  feasibility  of  incorporating 
interactive  maintenance-aiding  techniques  into  an  electronic  documentation 
interface  for  depot  maintenance.  The  advent  of  electronic  documentation  of 
technical  orders  (TOs)  through  ATOS  creates  both  a  need  and  an  opportunity. 

There  is  a  need  to  provide  the  technician  with  an  effective  interface  to  the  new 
medium.  There  is  also  an  opportunity  to  improve  the  form  of  information 
delivery  and  thus  maintenance  efficiency. 

The  study  focused  on  identifying  the  information  presentation  needs  of 
maintenance  technicians  and  determining  the  interface  requirements  and 
supporting  system  concepts  for  meeting  those  needs. 

Long-range  objectives  are  to  ensure  that  the  use  of  interface  and  aiding 
techniques  keeps  pace  with  the  use  of  advanced  information  storage, 
manipulation,  and  delivery  technologies.  Then  the  full  potential  of  electronic 
documentation  for  productivity  and  error  reduction  can  be  reached. 


1.2  SCOPE 


This  study  focused  on  the  requirements  for  an  interactive  maintenance-aiding 
system  (IMAS),  an  information  presentation  system  concept  suitable  for  depot 
maintenance  applications  and  compatible  with  ATOS,  which  is  under  development. 
The  study  also  assesses  the  feasibility  of  incorporating  advanced 
performance-aiding  techniques  into  depot  maintenance  activities  and  identifies 
the  potential  benefits  of  such  aiding. 

This  study  was  an  exploratory  effort  resulting  in  a  number  of  recommendations 
for  designing  and  implementing  a  system  for  creating  and  maintaining  electronic 
documentation  interfaces  for  maintenance  technicians.  The  study  did  not  involve 
the  design  or  construction  of  actual  systems. 

A  variety  of  maintenance  activities  at  the  Ogden  ALC  were  reviewed.  Information 
needs  for  job  performance  were  examined,  and  current  practices  were  compared  to 
those  needs.  Job  areas  that  could  benefit  from  application  of  an  IMAS  were 
identif ied. 

An  IMAS  concept  was  developed  to  meet  perceived  aiding  needs.  The  recommended 
technician-oriented  IMAS  will  be  an  adjunct  to  ATOS,  adapting  the  materials  from 
the  electronic  TO  documentation  to  job-specific  requirements  at  the  depot  level. 

Prototype  applications  of  interactive  aiding  systems  for  several  appropriate 
areas  are  recommended  along  with  plans  for  their  development. 

1.3  PROBLEM  OVERVIEW 

% 

1.3.1  Limitations  of  Conventional  Technical  Orders 

Thirty  years  of  research  on  job  performance  aiding  and  design  of  procedural  job 
guides  have  consistently  shown  that  less  trained  and  experienced  personnel 
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equipped  with  well-conceived  job  aids  can  perform  maintenance  tasks  at  the  same 
level  of  competence  as  more  highly  trained  and  experienced  technicians. 

Although  job  performance  aids  and  procedural  guides  have  not  been  widely  adopted 
as  part  of  the  maintenance  documentation  of  weapon  systems,  research  results 
have  had  a  discernible  effect  on  the  form  and  content  of  TOs.  For  instance,  the 
job  guides  for  the  F-16  Maintenance  Integrated  Data  Access  System  (MIDAS)  embody 
the  step-by-step  performance-aiding  concept. 

Unfortunately,  from  the  perspective  of  their  use  in  job  applications,  the 
improved  TOs  suffer  most  of  the  same  shortcomings  as  conventional  TOs.  These 
shortcomings,  listed  below,  are  frequently  noted  in  the  literature  (Reference 
1). 

•  Multiple  cross-referencing  is  required  both  within  and  between  volumes. 

•  Search  time  to  identify,  locate,  and  retrieve  relevant  portions  of 
manuals  is  long. 

•  Information  is  often  obsolete  or  inappropriate  as  a  result  of  delays  in 
updating. 

•  Information  is  often  difficult  to  understand  as  presented. 

•  The  medium  is  inflexible  in  adapting  to  the  skill  level  of  the 
technician. 

•  Volumes  are  large  and  bulky. 

•  Too  little  or  too  much  information  is  presented, 

MIDAS  is  a  large  step  forward  in  reducing  the  shortcomings  of  traditional  TOs. 

It  has  a  less  bulky  format  and  presents  procedures  in  two  levels  of  detail. 

However,  the  paper  format  of  these  volumes  precludes  a  real  solution  to  the 
problems  of  cross-referencing,  updating,  and  comprehensibility.  An  ideal  medium 
for  providing  technical  information  and  job  instructions  to  a  technician  should: 
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•  Provide  only  the  information  needed  for  the  task  at  hand 

•  Provide  current  and  automatically  cross-ref erenced  information 

•  Retrieve  all  information  by  entering  a  task 

•  Allow  graphic  and  pictorial  materials  to  accompany  textual  procedures 

•  Cross-index  related  items 

•  Offer  more  detailed  explanations,  definitions,  technical  context,  and 
procedural  rationale  on  request 

•  Present  information  on  a  display  that  does  not  require  looking  away  from 
the  task 

An  electronic  interface  with  integral,  interactive  aiding  can  satisfy  these 
requirements  by  achieving  the  real-time  manipulation  and  dynamic 
user- interactive  aiding  that  takes  full  advantage  of  the  job  aiding  concept  that 
MIDAS  reflects. 

1.3.2  Electronic  Documentation  under  ATOS 


Currently,  traditional  paper  TOs  used  in  military  aircraft  maintenance  are  being 
converted  to  electronic  documents  under  the  multiphase  program,  ATOS.  The  first 
phase  of  the  program  was  to  develop  a  computer  system  and  software  for 
preparing,  compiling,  and  printing  TOs.  The  first-phase  prototype  system  has 
been  installed  at  Ogden  ALC.  At  present,  only  depot-level  TO  changes  are  being 
entered  into  the  system  at  Ogden. 

The  objective  of  the  second  phase,  currently  in  progress,  is  to  provide  a  ground 
network  for  distribution  of  the  information  in  ATOS  to  maintenance 
organizations,  weapon  system  managers,  and  support  personnel  on  a  base.  The 
distribution  system  must  include  an  interface  in  each  organization  or  functional 
unit  by  which  the  users  of  TOs  can  retrieve  portions  of  the  technical  manuals 
relevant  to  specific  tasks  within  a  job. 

The  TOs  in  the  present  form  of  ATOS  are  conventional  paper  documents  that  have 
been  digitized  for  storage  and  processing  by  a  computer.  The  computer's  primary 
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delivery  function  at  this  point  is  to  access  user-designated  sections  of  a  TO. 
Such  systems  have  been  charact eriz ed  as  "electronic  page- turners"  in  the  sense 
that  the  basic  format  is  equivalent  to  that  of  traditional,  paper-based 
information  systems.  The  system  is  a  first  step  in  the  geneial  trend  toward 
electronic  documentation  and  the  elimination  of  paper;  the  system  electronically 
stores  documents  and  efficiently  produces  hard  copy.  Substantial  savings  will  be 
achieved  in  the  preparation,  printing,  and  distribution  of  manuals. 

Furthermore,  the  ease  and  speed  with  which  changes  in  a  TO  can  be  made  and 
distributed  will  improve  the  quality  of  maintenance  and  reduce  the  cost  of 
making  the  changes. 

1.3.3  Relationship  of  XMAS  to  ATOS 

Electronic  documentation  is  a  medium  that  can  be  configured  to  possess 
properties  significantly  different  from  the  basic  page-turning  functions  of  a 
paper-oriented  system  like  ATOS.  These  properties  can  provide  useful  interface 
options  in  format,  information  organization,  access,  and  the  mix  of  textual, 
graphic,  and  pictorial  materials.  From  the  user's  standpoint,  these  options 
offer  greater  ease  and  speed  in  retrieving  integrated  information  relevant  to 
the  user's  task.  These  advantages  are  achieved  with  access  techniques  that 
reduce  the  complexity  of  maintenance  instructions  by  presenting  only  what  is 
needed  or  requested.  Efficiency  can  be  further  increased  with  presentation 
techniques  that  reduce  the  cognitive,  memorial,  and  perceptual  demands  on  the 
technician. 

A  paper-oriented  system  like  ATOS  cannot  be  expected  to  deliver  the  kind  of 
electronic  documentation  described  above.  The  primary  purpose  of  ATOS  is  to 
provide  an  archive  for  the  engineering  and  logistic  information  for  a  weapon 
system  in  a  format  compatible  with  hard-copy  production.  Because  it  is  designed 
for  hard-copy  production,  ATOS  is  not  structured  for  the  production  of 
sophisticated  aiding  interfaces. 


5 


We  recommend  that  the  ATOS  program  not  be  overloaded  with  facilities  for 
producing  the  kinds  of  technician  interfaces  described  in  this  report.  Instead, 
a  companion  system  should  be  developed  for  this  purpose.  To  produce  an  optimal 
interface  for  maintenance  aiding,  paper-based  information  must  be  reorganized 
for  an  integrated  presentation  and  preferably  filtered  for  relevance  to  the 
particular  checkout,  diagnosis,  or  repair  tasks  being  performed.  The  ideal 
maintenance  interface  must: 

•  Display  only  those  TO  procedures  relevant  to  a  given  task 

•  Provide  easy  or  completely  transparent  cross-r ef erencing  between 
procedures 

•  Integrate  text  and  graphics 

•  Aid  in  the  easy  identification  of  parts  and  components  through  the  use 
of  highlighting  and  other  perceptual  aids 

In  its  present  form,  ATOS  is  not  capable  of  providing  the  maintenance  interface 
needed  by  depot-level  technicians.  To  take  advantage  of  the  new  medium,  the 
interface  should  entail  more  than  »~he  simple  electronic  retrieval  of  TOs,  as  in 
ATOS.  The  opportunity  exists  to  incorporate  a  variety  of  performance-aiding 
techniques  as  an  integral  part  of  the  maintenance  instructions  received  by  the 
technician.  This  aiding  should  increase  productivity  by: 

•  Reducing  the  time  spent  finding  relevant  sections  of  TOs 

•  Increasing  the  clarity  and  comprehensibility  of  the  procedural 
instructions 

•  Simplifying  the  maintenance  tasks 

•  Reducing  the  cognitive  demands  imposed  on  the  technician 
1.3.4  The  Need  for  Depot-Level  Work  Package  Authoring 

A  depot  maintenance  organization  should  have  the  local  capability  to  prepare 
depot  **work  packages1*  that  integrate  procedures,  information,  and  pictures  into 


6 


task  instructions  for  performing  specific  jobs.  These  work  packages  could  be 
based  on  material  retrieved  directly  from  ATOS  files.  However,  to  create  an 
adequate  interface  for  the  technician,  ATOS  material  will  need  to  be  modified, 
reformatted,  and  supplemented  with  state-of-the-art  electronic  aiding  techniques 
appropriate  to  the  task  and  technician. 

An  "authoring  system"  would  allow  depot  maintenance  managers  and  supervisors  to 
compose,  compile,  update,  and  maintain  these  work  packages.  The  authoring 
capability  should  be  independent  of,  but  compatible  with,  ATOS  files  and 
software.  It  must  permit  information  derived  from  weapon  system  experience  and 
local  maintenance  history  and  practices  to  be  added  to  the  work  package.  It 
should  also  alert  users  to  changes  in  TOs  so  that  they  can  infer  and  implement 
changes  in  work  packages — changes  that  cannot  be  made  at  the  ATOS  level. 

ATOS  sr  «uld  not  be  expected  to  provide  these  work  packages  or  the  software  for 
an  authoring  capability  that  would  provide  sophisticated  aiding  at  the 
interface.  Rather,  the  highest  priority  objectives  for  ATOS  are  to  compile, 
maintain,  and  distribute  an  engineering  and  logistic  archive.  The  technical 
manuals  are  a  reference  source  used  by  several  classes  of  people  in  addition  to 
maintainers:  supply  specialists;  transportation  specialists;  operators;  design 

and  service  engineers;  and  curriculum  planners,  trainers  and  students  (Reference 
2).  The  needs  of  these  users  require  a  complex  mixture  of  information  in  a 
variety  of  formats  not  necessarily  relevant  to  maintenance  activities. 

1.4  THE  TECHNOLOGY  OF  INTERACTIVE  MAINTENANCE  AIDING 

1.4.1  Maintenance-Aiding  System  Configurations 

Interactive  maintenance-aiding  systems  provide  real-time  support  to  maintenance 
technicians  performing  checkout,  fault  isolation,  and  repair  tasks.  They  can  be 
built  using  off-the-shelf  equipment  for  interactive  graphics,  video  and  audio 
storage,  and  voice  interaction.  These  systems  can  present  procedural  guidelines 
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and  supporting  information  to  the  technician  using  a  display  medium,  such  as  a 
visual  display  unit  (References  3  through  6).  Thus  instructions  can  be 
presented  directly  and  rapidly  in  a  nonintrusive  manner. 

Such  a  system  eliminates  retrieving  printed  copies  of  manuals,  locating  sections 
relevant  to  the  task  being  performed,  and  cross-referencing  within  and  among 
\  manuals.  Depending  on  the  capabilities  provided,  the  technician  can  be  presented 

with  a  list  of  task  steps  that  contains  technical  data  and  highlights 
significant  features  and  events.  A  short  video  segment  can  be  used  to 
demonstrate  a  procedure  that  the  technician  can  emulate. 

An  IMAS  can  be  custom-designed  to  fit  the  requirements  of  a  specific  application 
by  the  selection  of  different  implementation  options  in  display,  input,  and 
storage  media.  Display  options  include  the  CRT,  electroluminescent  panel,  liquid 
crystal,  and  helmet-mounted  display.  Input  can  be  made  by  keyboard,  touch 
screen,  and/or  voice  recognition.  Storage  mechanisms  include  hard  or  floppy 
disks  for  digitized  alphanumeric  text,  graphics,  and  data,  and  video  or  optical 
disks  for  pictorial  material  and  motion  pictures.  An  interactive  aiding  system 
can  be  adapted  to  the  conditions  of  each  application  by  selecting  appropriate 
options  in  each  category. 

The  flexibility  provided  by  these  options  offers  systems  that  can  reduce  the 
time  spent  in  nonproductive  and  disrupting  activities  and  thus  increase 
maintenance  productivity.  These  advantages  are  further  enhanced  when 
performance-aiding  techniques  are  incorporated.  A  nominal  or  baseline  system 
configuration  is  a  conventional  CRT  display  on  a  desk  or  benchtop  with  a 
standard  keyboard  or  touch  screen.  The  storage  medium  depends  on  the  quantity 
and  type  of  information.  This  configuration  is  adequate  for  a  large  range  of 
applications.  However,  a  small,  lightweight,  portable  microprocessor  would  be 
appropriate  when  the  technician  must  be  mobile,  as  in  flight-line  maintenance. 

Several  other  variations  are  appropriate  for  special  work  environments  or 
conditions.  For  instance,  retrieval  of  information  by  voice  commands  leaves  the 
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technicians  hands  free  to  manipulate  equipment  while  monitoring  a  display.  A 
laser  disk  can  provide  a  high-fidelity  motion  picture  of  an  expert  performing  a 
procedure  that  is  not  easily  or  adequately  described  by  words  alone.  Finally, 
if  the  display  is  helmet-mounted,  the  information  is  available  even  in  cramped 
spaces  where  there  is  no  room  for  a  TO  or  display  terminal. 


1.4.2  Criteria  and  Techniques  for  Aiding  Maintenance  Tasks 


There  are  two  places  for  storing  the  information  needed  by  the  maintenance 
technician  for  successful  problem  solving:  in  the  technician's  head  or  in 
external  sources,  such  as  technical  manuals.  Neither  form  is  completely 
adequate  for  supporting  maintenance  activities.  Human  memory  is  typically  not 
adequate  given  limited  training  resources  and  experience.  Information  retrieved 
from  memory  tends  to  be  incomplete,  inaccurate,  and  unreliable.  The  number  of 
maintenance  procedures,  the  amount  of  task-specific  data,  and  the  possibly 
infrequent  performance  of  any  single  maintenance  task  force  the  techician  to 
rely  on  external  sources. 


1.4. 2.1  Limitations  of  Paper  Aiding — Unfortunately,  the  best  available  manuals, 
represented  by  the  conventional  TO  system,  are  also  inadequate  for  supporting 
the  technician's  performance.  Interactive  maintenance  aiding  is  an  approach  to 
providing  a  supplementary  form  of  documentation  that  can  overcome  the 
limitations  of  the  technician's  memory  and  the  conventional  manual. 


Table  1  presents  a  summary  of  aiding  techniques.  Aiding  solutions  are  presented 
in  conjunction  with  task  requirements.  In  general,  task  requirements  can  impose 
excessive  cognitive  demands  on  the  technician  in  the  form  of  short-term  and 
long-term  memory,  perceptual  filtering  of  visual  information,  selective 
attention,  interpretation,  and  inferencing.  Table  1  presents  both  electronic 
and  paper  solutions.  Electronic  solutions  utilize  recent  advances  in  computer 
technology,  especially  in  interactive  graphics,  windowing  techniques,  and 
digitized  pictorial  imagery.  Paper  solutions  are  generally  a  subset  of  the 
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electronic  methods.  Electronic  aiding  is  broader  in  coverage,  more  interactive, 
faster  in  execution,  and  more  convenient  to  use  in  real  time. 

Paper  aiding  is  an  approach  that  has  been  explored  by  the  armed  services.  The 
PIMO  program  (Presentation  of  Information  for  Maintenance  and  Operation) 
established  the  characteristics  of  the  presentation  format  for  current  paper 
aiding  techniques  (Reference  1).  These  characteris tics  include: 

•  Relevant  and  complete  information  presented  only 

•  Procedures  broken  down  to  the  task  level 

•  Standardized  sentence  structure  and  line  length 

•  Improved  access  to  information  through  indexing 

These  characteristics  have  been  shown  to  be  effective  in  reducing  total  elapsed 
time  and  maintenance  errors  (Reference  7).  However,  their  effectiveness  is 
limited  by  the  presentation  medium.  Paper  manuals  are  unable  to  present  only 
the  information  appropriate  to  the  skill  level  of  the  technician  for  the  task  at 
hand.  The  use  of  job  guides  and  fault  isolation  manuals  are  attempts  to 
overcome  this  problem,  but  often  they  present  too  little  or  too  much 
information.  This  problem,  combined  with  the  growing  size  and  number  of 
technical  manuals,  increases  the  time  required  to  access  relevant  information, 
thus  degrading  the  utility  of  manuals. 

1.4. 5. 2  Capabilities  of  Electronic  Aiding — An  electronic  aid  can  accommodate 
all  of  the  paper  aiding  techniques  and  incorporate  new  technologies  to  overcome 
the  limitations  of  current  TOs.  Cross-referencing  can  be  performed 
automatically,  for  example,  and  information  can  be  presented  at  any  desired 
level  of  detail.  The  use  of  paper  manuals  is  made  obsolete  as  the  technician 
can  access  information  by  calling  up  a  specific  avionics  system.  "Hypertext'1  is 
a  means  for  obtaining  definitions,  explanations,  or  more  detailed  information. 
Supporting  information  can  be  accessed  by  key  word  selection,  allowing  the 
operator  to  display  additional  details  only  when  needed. 
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A  range  of  possible  solutions  is  usually  available  for  a  given 
performance-aiding  problem.  Once  the  need  for  an  information  aiding  system  is 
demonstrated,  the  specific  design  details  can  be  developed  through  the 
examination  of  task  requirements  listed  in  Table  1.  For  example, 
trouble-shooting  tasks  place  a  demand  on  the  short-term  memory  of  the 
technician,  who  must  simultaneously  track  different  types  of  information,  such 
as  the  location  of  a  part,  circuit  diagrams,  block  diagrams,  and  specific  data 
(e.g.  voltage  readings).  The  job  guide  system  uses  pictorial  locators,  keyed 
schematics,  and  indexing  to  ease  the  integration  burden,  but  it  forces  the 
technician  to  remember  several  pages  of  information  from  one  or  more  manuals. 

An  IMAS  can  provide  all  of  these  capabilities  and  allow  simultaneous  viewing  of 
these  sources  through  the  use  of  windowing  or  split  screen  techniques.  The  use 
of  animation  further  reduces  this  burden.  For  example,  the  technician  can  view 
the  path  of  electrical  flew  in  a  circuit  diagram.  Video  disks  can  provide 
instruction  for  tasks  requiring  integrated  procedures,  such  as  those  that  cannot 
be  explained  by  traditional  alphanumeric  presentation. 

An  electronic  information  aid  can  also  reduce  perceptual  demands  on  the 
technician.  Paper  information  aiding  solutions  to  perceptual  demands  include 
boldfacing,  photographs,  illustrations,  and  highlighting.  These  aid  the 
technician  in  picking  out  information  from  the  manual,  identifying  parts  on  the 
aircraft,  and  viewing  overall  relationships  between  components  or  systems.  An 
electronic  aid  can  provide  these  solutions  as  well  as  information  filtering, 
which  tailors  the  information  to  the  specific  skill  level  of  the  technician.  In 
addition,  digitized  photographs  can  be  rotated  or  expanded  so  that  the 
technician  can  receive  contextual  information  for  part  location  and  component 
relationships. 

If  the  task  requires  extensive  attention  to  detail,  a  paper  aid  can  provide  a 
checklist  of  procedures  for  the  technician.  However,  this  produces  additional 
pages  of  documentation.  The  technician  must  integrate  than  and  keep  place  in 
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the  checklist.  An  electronic  checklist  can  present  a  limited  number  of  steps  at 
a  time  and  demand  that  the  operator  check  off  each  step  before  the  next  step  is 
presented.  Related  information  from  other  sources  can  be  easily  integrated 
automatically.  For  example,  if  a  specific  instruction  on  a  checklist  is 
familiar,  the  maintenance  technician  can  perform  the  task  and  indicate  to  the 
system  that  it  is  finished.  If  an  instruction  is  not  familiar,  the  technician 
could  request  additional  instructions  from  the  system. 

Electronic  aiding  can  improve  aircraft  repair  tasks  that  require  the  technician 
to  perform  activities  in  restricted,  cramped  spaces  or  in  areas  that  are  not 
near  the  location  of  the  TOs.  These  situations  normally  increase  the 
maintenance  repair  time  by  requiring  the  technician  to  make  several  trips  to  the 
TO  outside  of  the  work  space  or  by  requiring  an  additional  technician  to 
retrieve  TOs  and  find  the  necessary  information.  The  latter  activity  is  also 
necessary  when  the  job  requires  the  technician  to  use  both  hands.  Advances  in 
hardware  for  electronic  aiding  systems,  such  as  portable  display  units, 
helmet-mounted  displays,  touch  input,  and  voice  input /output,  can  alleviate 
these  problems  by  bringing  the  information  source  to  the  task  area  and  allowing 
access  of  relevant  data  with  little  or  no  use  of  the  technician's  hands. 

1.4,3  An  Example  of  Interactive  Electronic  Aiding 

An  example  of  electronic  aiding  will  be  presented  here  for  the  job  of 
calibrating  a  radar  using  radar  test  set  AN/APM  383,  This  set  is  a  special 
purpose,  portable  tester  used  to  maintain  continuous  wave  illuminator  radar 
equipment.  It  measures  RF  power,  spectral  purity,  and  coding  integrity  of 
signals  generated  by  the  radar  equipment. 

There  are  five  technical  manuals  written  for  this  equipment:  operation 
instructions  (TO  33D7-44-21 9-1 ) ,  calibration  procedures  (TO  33D7-44-21 9-2) , 
intermediate  maintenance  instructions  (TO  33D7-44-21 9-12) ,  overhaul  instruct ious 
(TO  33D7-44-21 9-4) ,  and  illustrated  parts  breakdown  (TO  33D7-44-21 9-4) . 
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The  electronic  aiding  approach  consists  of  integrating  the  information  from 
these  manuals  into  one  procedure,  to  be  accessed  by  menu  selection.  Figure  1 
presents  a  sample  menu  structure  for  the  overhaul  instructions.  The  integrated 
information  was  organized  into  a  menu  structure  so  that  the  user  can  obtain 
information  at  increasing  levels  of  detail.  Each  manual  becomes  an  item  on  the 
highest  level  menu  for  the  radar  test  set.  Once  the  desired  level  of 
information  is  selected,  the  technician  can  choose  between  procedural 
instructions  and  supporting  information.  Both  categories  are  broken  down 
through  three  levels,  terminating  at  bLock  and  circuit  diagram  descriptions 
(supporting  information)  and  procedural  steps  for  test  set  up  and  test 
(procedures) . 

Figures  2  through  6  present  sample  interfaces  based  on  this  menu  structure  using 
a  CRT  with  windowing  and  a  touch  screen.  These  figures  present  a  possible 
sequence  of  information  retrieval  for  a  technician  performing  the  ninth  test 
step  of  the  checkout  procedures  for  voltage  regulator  assembly  A1A22. 
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The  displayed  instructions  for  this  test  step  are  presented  in  Figure  2.  Test 
procedures  are  presented  on  the  left  portion  of  the  screen  and  results  and 
abnormal-action  directions  on  the  right.  Key  words  are  used  to  allow  operator 
access  to  additional  information  on  this  procedure.  Touching  any  of  the 
uppercase  words  produces  a  menu  of  available  information  categories  on  the  lower 
portion  of  the  screen.  In  this  example,  the  technician  touches  "Q5"  to  obtain 
more  information  on  that  component. 


Figure  2.  Display  for  the  Ninth  Test  Step  in  Checkout  of  Voltage 
Regulator  Assembly  A1A22 
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The  request  for  more  information  on  Q5  is  follcved  by  the  display  shown  in 
Figure  3.  A  menu  of  available  information  on  this  component  appears  at  the 
bottom  of  the  display-  When  ''CIRCUIT  DIAGRAM"  is  selected,  the  display  changes 
to  the  circuit  diagram  presented  in  Figure  4;  the  right  half  of  the  screen  is 
blank- 


Figure  3-  Display  of  Menu  on  Bottom  of  Screen  to  Call  for  More  Information; 
Evoked  by  Touching  "Q5"  in  the  Upper  Part  of  the  Screen 
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Touching  "QS"  on  the  circuit  diagram  results 
on  the  right  half  of  the  screen  in  Figure  4. 
bottom  of  the  screen*  "CIRCUIT  DESCRIPTION" 
part  of  the  screen. 


in  the  display  of  information  on  Q5 
The  menu  also  appears  again  at  the 
is  now  selected  by  touching  that 


Figure  4.  Display  of  Circuit  Diagram  Containing  Q5 ;  Technician  Requests 
More  Information  by  Touching  "CIRCUIT  DESCRIPTION"  at  Bottom 
of  Screen 
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The  display  on  the  left  side  of  the  screen  in  Figure  5  is  a  diagram  of  the 
circuit  containing  Q5  and  associated  components.  Control  commands  appear  in  the 
lower  left;  the  technician  would  use  them  for  scrolling  the  diagram.  In  this 
example,  the  technician  desires  more  information  on  this  component  and  touches 
,fQ5 '*  on  the  circuit  diagram. 


Figure  5.  Display  of  Circuit  Diagram  Containing  Q5  with  Symbols  for  Scrolling 
Commands;  Touching  "Q5"  Again  Calls  for  More  Information 
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The  circuit  diagram  appears  on  the  left  half  of  the  screen  in  Figure  6, 

Commands  for  viewing  additional  information  or  recalling  previous  information 
are  also  available. 

This  system  would  provide  easy  access  to  all  information  available  on  the  radar 
test  set. 


Figure  6.  Presentation  of  Requested  Circuit  Information  and 
Symbols  for  Scrolling  Commands 
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SECTION  2 


REVIEW  OF  OGDEN  ALC  MAINTENANCE  ACTIVITIES 


The  basic  objectives  of  this  study  were  to  assess  the  feasibility  of 
implementing  an  interactive  maintenance-aiding  system  (IMAS)  for  depot 
maintenance  tasks  and  to  develop  a  roadmap  for  developing  and  installing  such  an 
ATOS-compa tible  system  at  Ogden  ALC.  If  feasible  applications  were  found,  then 
the  development  required  to  design  and  build  a  limited  prototype  for  test  and 
evaluation  was  to  be  identified.  This  developmental  work  was  to  be  described  at 
a  level  necessary  to  set  up  a  program  plan  and  provide  a  basis  for  preparing 
requests  for  proposals  to  develop  the  prototype (s ) . 

2.1  APPROACH 


The  approach  consisted  of  five  steps. 

1.  Collect  information  on  maintenance  tasks  at  Ogden  ALC. 

2.  Identify  candidates  for  IMAS  application  in  job  areas  that  offer 
potential  benef its. 

3.  Recommend  a  limited  set  of  applications  for  system  development. 

4.  Select  and  describe  IMAS  prototypes  for  development. 

5.  Outline  a  developmental  roadmap  for  installing  an  IMAS  at  Ogden  ALC. 

Valuable  sources  of  information  were  interviews  and  observations  of  maintenance 
activities  made  during  three  visits  to  Hill  Air  Force  Base  (AFB).  Discussions 
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were  held  with  planners,  supervisors,  and  experienced  technicians  in  several 
shops,  facilities,  and  laboratories*  Relevant  descriptive  literature  and  TOs 
were  obtained  and  reviewed. 

Candidate  job  areas  were  surveyed  to  identify  those  that  are  most  dependent  on 
TOs  for  successful  performance  and  therefore  offer  significant  potential  benefit 
for  aiding.  The  criteria  listed  in  Table  1  were  used  to  determine  those 
maintenance  areas  with  the  highest  need  for  information  aiding.  Although  TOs 
are  required  for  all  maintenance  activities,  some  tasks  are  more  dependent  on 
them  than  others.  In  this  investigation,  the  most  critical  task  requirements 
were  those  demanding  long-term  memory  and  visual  perception.  A  first  cut  was 
made  at  determining  information  requirements,  desirable  aiding  techniques,  and 
an  aiding  system  concept.  Candidate  areas  selected  for  further  development  were 
then  examined  in  greater  detail  to  obtain  additional  information  on  the  use  of 
relevant  TOs. 

In  addition,  information  was  gathered  on  related  programs:  ATOS;  the  Integrated 
Maintenance  Information  System  (IMIS),  a  developmental  program  at  the  Air  Force 
Human  Resources  Laboratory  (AFHRL)  at  Wright-Pat terson  AFJ ;  and  the  Flight 
Control  Maintenance  Diagnostic  System  (FCMDS),  a  Wr ight-Pa tter son  rlight 
Dynamics  Lab  program  for  the  development  of  an  expert  diagnostic  system  for 
fault  isolation  for  aircraft  maintenance.  Information  from  subsequent  visits 
with  key  people  in  these  programs  and  from  descriptive  documents  were  used  to 
further  formulate  an  integrated  IMAS  concept. 

2.2  SURVEY  OF  DEPOT  WORK  AREAS 

Five  maintenance  areas  were  visited. 

•  Aircraft  Repair  and  Modification  Facility — Overhaul,  repair,  and 

modification  of  F-4,  F-15,  and  F-16  aircraft  are  done  in  this  facility. 
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•  Precision  Measurement  Equipment  Laboratory  (PMEL) — This  organization 
consists  of  seven  sections  that  inspect,  calibrate,  and  overhaul  all 
measurement  equipment  used  at  Hill  AFB.  The  observations  and  analyses 
in  this  study  of  PMEL  were  focused  on  the  Electro  Mech/Optical 
Dimensional  Section. 

•  Landing  Gear  Division — This  organization  is  an  industrial  facility  for 
overhaul,  repair,  and  reconditioning  of  all  components  of  the  landing 
gear  (such  as  wheels,  brakes,  and  associated  hydraulics  of  the  aircraft) 
in  the  Air  Force  inventory. 

•  Sheet  Metal  Shop — Structural  metal  repair  and  modification  are  done  in 
this  shop.  A  modification  to  all  F-16  wing  assemblies  was  being 
performed  at  the  time  of  our  visit. 

•  Automatic  Test  Equipment  (ATE)  Facility — Fault  localization  by  the 
operator  after  the  equipment  has  narrowed  the  location  of  the  fault  to  a 
limited  number  of  components  is  an  area  of  potential  IMAS  application. 

Conversations  were  held  with  technicians,  managers,  and  planners  at  each 
facility  to  determine  what  work  is  done,  how  it  is  done,  and  where  TOs  and  other 
information  sources  are  needed  and  used.  The  objective  of  this  information 
gathering  was  to  provide  information  in  the  following  categories: 

•  The  flow  of  information  in  the  facility 

•  Maintenance  tasks  suitable  for  the  initial  prototyping  of  IMAS 

•  Limitations  of  the  TOs  for  each  task 

•  How  the  TOs  are  used  by  technicians,  planners,  and  managers 

•  Other  sources  of  information  required  for  successful  task  performance 

•  Recommendations  for  IMAS  design 
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Maintenance  activities  were  also  observed  at  each  facility.  In  the  F-16 
modification  area,  we  took  a  guided  'Valk  through”  of  the  facility  from  planning 
to  execution  of  modif ica tions  on  the  hangar  floor  and  into  system  checkout  for 
flight  test.  Installation  of  an  electrical  generator  and  the  associated  power 
distribution  wiring  was  taking  place  at  this  time. 

In  addition,  TOs  were  obtained  for  the  job  areas  that  were  candidates  for 
prototyping.  These  were  examined  for  task  descriptions,  type  of  procedures,  use 
of  supporting  information,  cross-referencing  of  tasks,  potential  for  performance 
aiding,  level  of  detail,  expertise  required  for  use,  and  illustrations. 

2.3  SUMMARY  OF  OBSERVATIONS  FROM  THE  REVIEW  OF  MAINTENANCE  ACTIVITIES 

The  following  is  a  brief  summary  of  our  conclusions  on  candidate  task  areas. 

•  Flight  test  checkout  following  F-16  modifications  can  benefit  from  a 
time  savings  of  up  to  30%  by  integrating  multisource  technical 
information  into  work  packages  for  presentation.  Additional  savings  can 
be  obtained  with  electronic  presentation  techniques. 

•  PMEL  can  benefit  from  multisource  integration  and  memory  aiding  for 
lengthy  and  infrequently  performed  tasks.  Perceptual  aiding  will  also 
benefit  tasks  involving  unfamiliar  or  infrequently  maintained  equipment. 

•  The  tasks  at  the  Landing  Gear  Division  and  Sheet  Metal  Shop  will  not 
benefit  enough  from  aiding  to  merit  further  investigation.  The  tasks 
are  highly  structured  and  frequently  performed  so  that  they  are 
memorized  and  mastered  even  when  lengthy  and  complex.  Real-time 
interactive  use  of  TOs  is  not  routinely  required.  Interactive  aiding  is 
a  low  priority  and  would  have  little  benefit. 
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2.4  DISCUSSION  OF  WORK  AREA  ACTIVITIES 


2*4.1  Aircraft  Repair  and  Modification  Facility 


Overhaul,  repair,  and  modification  of  F-4,  F-15,  and  F-16  aircraft  are  done  in 
the  Aircraft  Repair  and  Modification  Facility.  Periodic  overhaul  is  a  major 
activity  in  the  F-4  and  F-15  areas.  These  aircraft  are  mature  weapon  systems 
and  are  in  the  midphase  of  their  life  cycle.  While  IMAS  would  be  benefical,  the 
extent  of  benefit  and  the  projected  longevity  of  the  F-4  and  F-15  reduce  the 
priority  of  these  weapon  systems  as  vehicles  for  prototype  development  and 
evaluation. 

The  F-16,  in  contrast,  is  a  relatively  new  system.  Its  design  configuration  and 
maintenance  practices  are  still  rapidly  evolving.  The  main  activity  of  the  F-16 
facility  is  performing  the  aircraft  modifications  dictated  by  a  continuing 
series  of  engineering  changes.  Each  F-16  in  the  Air  Force  inventory  is 
scheduled  to  be  updated  with  all  engineering  changes  that  have  been  made  since 
the  aircraft  was  produced.  No  repair  or  overhaul  is  done  except  for  rebuilding 
aircraft  damaged  in  accidents  or  making  repairs  that  cannot  be  made  at  the 
organizational  level. 

2. 4. 1,1  F-16  Maintenance  Problems — The  F-16  maintenance  tasks  are  fraught  with 

information  delivery  problems  because  of  both  the  complexity  of  modern  avionics 
systems  and  the  large  number  of  modifications  made  to  the  F-16.  Because  it  is 
unlikely  that  any  two  F-16  planes  will  have  the  same  configuration,  maintenance 
procedures  are  different  for  each  airplane  and  there  is  a  large  number  of 
possible  procedural  combinations.  The  necessary  cross-correlation  among 
modifications,  aircraft  models,  and  procedures  is  time-consuming,  complex,  and 
causes  a  significant  cognitive  demand  on  the  technician. 

There  are  four  different  production  models  of  the  aircraft.  The  later  models 
come  off  the  production  line  with  all  the  modif ica tions  of  earlier  models.  Over 
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800  modifications  had  been  made  at  the  time  of  our  review.  For  a  given  plane, 
some  subset  of  these  modifications  may  have  been  done  in  previous  visits  to  the 
repair  facility.  When  a  plane  arrives  at  the  facility,  a  maintenance  history 
record  of  that  aircraft  must  be  consulted  to  determine  its  modification  status. 

Further  discussion  of  the  cross-correlation  problem  requires  an  understanding  of 
Time  Ccxnpliance  Technical  Orders  (TCTOs).  A  modification  is  made  to  an  aircraft 
in  response  to  a  TCTO.  This  TCTO  contains  step-by-step  instructions  for 
removing  panels  and  components  to  get  to  the  site  of  the  modification, 
performing  the  modification,  and  then  reassembling  the  aircraft.  Each 
modification  has  a  procedure  designed  specifically  for  that  job.  Often  the  work 
is  done  using  blueprints  rather  than  the  TO  manuals.  However,  if  a  required 
procedure  is  in  the  job  guide,  the  TCTO  will  refer  to  the  TO  rather  than  repeat 
the  procedure.  Once  the  procedure  has  been  set  up,  the  modification  will  be 
made  to  all  F-16s  coming  into  the  facility  until  the  entire  fleet  has  been 
serviced.  The  procedure  changes,  however,  as  experience  is  gained  in  making  a 
particular  modification. 

Table  2  is  a  page  from  an  index  used  to  cross-reference  accomplished  TCTOs  and 
aircraft  serial  numbers  to  maintenance  procedures.  The  TCTOs  are  indicated  by 
number;  another  table  contains  a  description  of  the  modification  performed  under 
each  TCTO.  The  numbers  under  each  TCTO  an-  the  serial  numbers  (tail  numbers)  of 
the  aircraft  on  which  the  modification  has  been  made.  The  white  "A"  or  "B"  on 
black  background  indicates  the  model  of  the  craft.  The  white  numbers  on  black 
are  a  cross-reference  code  to  the  maintenance  procedures.  The  procedures 
associated  with  each  modification  are  interlaced  in  the  manual. 

The  technician  comes  to  critical  points  in  each  procedure  at  which  one  of  two 
variants  in  the  procedure  applies  to  the  aircraft  model  being  modified.  Each 
variant  will  be  marked  with  a  white  number  on  black  background.  The  technician 
then  refers  to  the  list  in  the  front  of  the  manual  to  determine  if  the  aircraft 
has  been  modified.  The  manual  from  which  Table  2  was  taken  has  eight  pages  of 
this  coding  dealing  with  13  TCTOs  for  the  flight  control  system. 
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TABLE  2.  CROSS-REFERENCING  OF  TCTOs  AND  AIRCRAFT 


T.O.  lM6A-2-27F»-00-l 

• 

USAF 

o  S/N  79-0332 

80-0540 

USAF 

to  S/N  78-0077 

- 

78-0098 

EPAF 

0  S/N  78-0141 

— 

80-3546 

EPAF 

O  S/N  78-0162 

— 

78-0167 

78-0189 

— 

78-0203 

78-0204 

- 

78-0205 

78-0238 

— 

78-0257 

78-0259 

- 

78-0264 

78-0285 

-* 

78-0299 

78-0301 

— 

70-0302 

USAF 

O  S/N  79-0420 

— 

80-0634 

EPAF 

0  S/N  78-0172 

-* 

78-0173 

• 

USAF 

®  S/N  78-0022 

- 

79-0331 

78-0209 

- 

78-0211 

EPAF 

O  S/N  78-0133 

- 

78-0140 

78-0267 

78-0271 

78-0177 

- 

78-0188 

78-0305 

— 

78-0307 

78-0224 

- 

78-0237 

AND 

78-0275 

- 

78-0284 

After  TCTO  1F-16-780 

USAF 

O  S/N  78-0001 

- 

79-0331 

USAF 

(I  S/N  78-0099 

- 

79-0419 

EPAF 

O  S/N  78-0116 

- 

78-0140 

EPAF 

a  S/N  78-0168 

- 

78-0171 

78-0174 

- 

78-0188 

78-0206 

— 

78-0208 

78-0212 

- 

78-0237 

78-0285 

— 

78-0266 

78-0272 

- 

78-0284 

78-0303 

— 

78-0304 

AND 

USAF 

o  S/N  78-0077 

— 

79-0419 

After  TCTO  1F-16-591 

EPAF 

m  S/N  78-0162 

— 

78-0171 

USAF 

Q  S/N  78-0001 

- 

78-0021 

78-0204 

- 

78-0208 

EPAF 

a  S/N  78-0116 

— 

78-0132 

78-0259 

— 

78-0266 

78-0174 

— 

78-0176 

78-0301 

- 

78-0304 

78-0212 

78-0223 

78-0272 

— 

7843274 

m 

Prior  to  TCTO  IF-16-591  and  LF-16-645 

USAF 

o  S/N  78-0001 

- 

78-0021 

USAF 

a  S/N  78-0077 

— 

78-0098 

EPAF 

o  S/N  78-0116 

— 

78-0132 

EPAF 

O  S/N  78-0162 

- 

78-0167 

78-0174 

— 

78-0176 

78-0204 

- 

784)205 

78-0212 

- 

78-0223 

78-0259 

- 

78-0264 

78-0272 

— 

78-0274 

78-0301 

- 

784)302 

USAF 

O  S/N  78-0077 

- 

78-0098 

EPAF 

O  S/N  78-0162 

— 

78-0167 

O 

USAF 

Q  S/N  78-0022 

- 

78-0204 

— 

78-0205 

EPAF 

o  S/N  78-0133 

- 

78-0259 

— 

78-0264 

78-0177 

— 

78-0301 

78-0302 

78-0224 

— 

78-0265 

— 

USAF 

O  S/N  78-0022 

- 

80-0540 

EPAF 

O  S/N  78-0133 

- 

80-3546 

USAF 

O  S/N  78-0099 

- 

78-0177 

— 

78-0203 

EPAF 

a  S/N  78-0168 

- 

78-0224 

— 

78-0257 

78-0206 

— 

78-0275 

7eH>299 

78-0265 

— 

78-0303 

-» 

USAF 

O  S/N  78-0099 

- 

80-0634 

AND 

EPAF 

o  S/N  78-0168 

— 

78-0173 

After  TCTO  1F-16-591  and  lF-16-645 

78-0206 

— 

78-0211 

USAF 

o  S/  N  78-0001 

— 

78-0021 

78-0265 

— 

78-0271 

EPAF 

o  S/N  78-0116 

— 

78-0132 

78-0303 

- 

78-0307 

"8-0174 

— 

78-0176 

AND 

78-0212 

— 

78-0223 

After  TCTO  IF-16-591 

78-0272 

— 

78-0274 

USAF 

m  S/N  78-0001 

** 

78-0021 

EPAF 

o  S/N  78-0116 

78-0132 

USAF 

o  S/N  78-0077 

— 

78-0098 

78-0174 

— 

78-0176 

EPAF 

O  S/N  78-0162 

— 

78-0167 

78-0212 

— 

78-0223 

78-0204 

- 

78-0205 

78-0272 

- 

78-0274 

78-0259 

— 

78-0264 

78-0301 

78-0302 

A/B  xiv 
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We  were  told  by  a  maintenance  supervisor  that  up  to  502  of  the  technician1 s  time 
during  checkout  and  trouble-shooting  following  modifications  is  spent  in 
paper-based  information  search.  Other  sources  report  that  302  of  total  job  time 
is  consumed  by  this  activity  (References  8  and  9).  Therefore,  improving  the 
methods  of  obtaining  and  delivering  information  is  critical  for  efficiency. 
Cross- indexing  information  between  procedures  and  across  and  within  subsystems 
is  a  crucial  part  of  the  IMAS  concept. 

The  maintenance  tasks  on  the  F-16  are  also  made  more  difficult  by  the  fact  that 
the  F-16  subsystems  are  more  interdependent  than  prior  aircraft.  The  use  of 
sophisticated  electronics  to  achieve  the  high  performance  capability  of  the 
weapon  system  requires  a  great  deal  of  crosstalk  and  integration  among 
subsystems  that  traditionally  have  functioned  with  a  high  degree  of 
independence.  The  consequence  for  maintenance  is  that  symptom  interpretation  is 
more  complex.  Negative  indicators  during  checkout  and  fault  localization  can  no 
longer  be  interpreted  in  terms  of  a  single  subsystem.  The  interactions  between 
subsystems,  such  as  weapons  and  flight  control,  must  also  be  considered. 

2. 4. 1,2  The  F-16  Modification  Process — The  process  involved  in  aircraft 
modification  can  be  divided  into  three  phases:  planning,  production,  and  flight 
test.  P  ling  consists  of  generating  a  work  plan,  a  sequence  of  activities, 

and  a  sc  ale.  Production  consists  of  the  activities  of  performing  the 

modif ica tion.  Flight  test  is  an  operational  checkout  of  the  aircraft  after  the 
modification  has  been  accomplished  to  verify  that  the  aircraft  has  been  restored 
to  full  operational  capability.  This  sequence  is  described  in  the  following 
subsections. 


2. 4. 1,2.1  Planning — When  the  planning  department  is  informed  that  an 
aircraft  is  scheduled  for  modification,  it  enters  chat  aircraft  into  the 
maintenance  schedule.  The  planners  determine  which  modifications  are  required 
by  consulting  the  Aircraft  Maintenance  System  (ACMS),  This  system  contains  a 
list  of  all  F-16  tail  numbers  and  the  modif ica tions  that  have  been  installed  for 
each. 
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All  documents  related  to  these  modifications  are  collated.  They  typically  fill 
two  drawers  of  a  standard  file  cabinet.  This  collation  is  currently  done 
manually. 

The  required  modifications  are  entered  into  the  on-site  CYBER  system,  which 
creates  a  set  of  job  cards  for  the  aircraft.  These  cards  list  the  modif ica tions 
that  need  to  be  installed,  reference  the  appropriate  TO  information,  and 
sequence  the  modification  procedures. 

2. 4. 1.2. 2  Production — The  job  cards  with  supporting  information  are  sent  to 
the  production  maintenance  foreman,  who  typically  supervises  the  work  on  thre^ 
aircraft.  Each  card  provides  the  schedule  and  sequence  for  the  jobs  as  well  as 
references  to  the  job  guides  containing  relevant  procedures.  The  cards  contain 
the  grade  of  technician  required  for  each  job,  and  the  foreman  assigns  personnel 
to  the  aircraft  on  this  basis. 

When  the  modifications  are  completed,  the  plane  is  reassembled.  The  technicians 
check  out  all  systems  in  the  aircraft,  trouble-shoot,  and  make  any  necessary 
repairs. 

2. 4. 1.2. 3  Flight  Test — The  aircraft  is  then  sent  to  the  flight  test 
facility  where  the  systems  are  rechecked  and  the  plane  is  flight  tested.  If 
malfunctions  surface,  the  fault  is  isolated  and  the  necessary  repairs  or 
adjustments  are  made.  If  the  plane  passes  the  preflight  inspection  and  flight 
test,  it  is  returned  to  its  unit.  Otherwise,  it  is  returned  to  the  operations 
checkout  f acil ity . 

2.4,1 .3  MIDAS — The  primary  source  of  procedural  information  for  F-16 
maintenance  is  the  Maintenance  Integrated  Data  Access  System  (MIDAS).  It 
consists  of  256  volumes  covering  37  subsystems.  Six  different  types  of  volumes 
will  be  described.  They  are  a  representative  set  to  which  a  technician  might 
refer  in  checking  out,  trouble-shooting,  and  repairing  an  aircraft.  This  set 
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includes  the  Job  Guide  Manual,  Fault  Isolation  Manual,  Wiring  Diagram  Manual, 
Illustrated  Parts  Breakdown,  General  Vehicle  Description,  and  General  System 
Manual . 

2. 4*1. 3.1  Job  Guide  Manual — The  Job  Guide  Manual  provides  step-by-step 
instructions  for  all  commonly  performed  tasks  except  trouble-shooting.  There 
are  several  guide  books  for  each  aircraft  subsystem.  They  contain  preliminary 
information,  procedural  instructions,  and  illustrations  on  fold-out  pages. 
Instructions  are  presented  at  two  levels  of  detail.  Major  steps  or  subtask 
headings  (e.g. ,  "Remove  the  gear  box")  are  presented  in  boldface  followed  by 
detailed  instructions  in  standard  typeface.  The  experienced  technician  who 
remembers  the  detailed  procedure  may  use  only  the  boldfaced  steps.  If  the 
procedure  is  checkout,  references  to  the  trouble-shooting  manuals  are  provided 
for  each  test  that  fails, 

2. 4. 1.3. 2  Fault  Isolation  Manual — The  Fault  Isolation  Manual  contains 
trouble-shooting  information  in  logic  tree  format.  The  technician  first 
performs  a  system  checkout  using  the  job  guide.  When  a  malfunction  is 
identified,  the  job  guide  provides  a  code  number  that  refers  the  technician  to  a 
page  and  paragraph  in  the  Fault  Isolation  Manual.  The  logic  tree  provides 
step-by-step  instructions  for  isolating  a  fault.  The  outcome  at  each  step 
determines  the  next  step  to  be  taken,  and  the  procedure  is  followed  until  the 
fault  is  identified. 

If  the  fault  is  identified,  a  reference  is  given  to  the  job  guide  that  provides 
instructions  for  replacing  the  faulty  component.  However,  if  the  fault  is  not 
identified,  the  technician  will  be  instructed  to  refer  to  the  wiring  diagram  for 
further  trouble-shooting.  At  this  point,  technicians  must  devise  their  own 
trouble-shooting  procedures. 

2. 4. 1.3. 3  Wiring  Diagram  Manual — The  Wiring  Diagram  Manual  provides 
diagrams  of  all  aircraft  wiring  and  drawings  showing  the  locations  of 
connectors,  ground  points,  terminal  boards,  and  splice  areas. 
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2. 4.1* 3 .4  Illustrated  Parts  Breakdown — The  Illustrated  Parts  Breakdown 
contains  illustrations  of  all  aircraft  components.  It  also  provides  the 
location  in  the  aircraft,  the  part  number,  and  a  brief  description  for  each 
component.  A  second  volume  provides  a  numbered  list  of  parts  used  as  an  index 
to  the  illustrations. 

2. 4. 1.3. 5  General  Vehicle  Description — The  General  Vehicle  Description 
provides  a  general  description  of  the  aircraft. 

2. 4. 1.3. 6  General  System  Manual — The  General  System  Manual  provides  a 
general  description  of  the  major  avionics  systems  (e.g.,  flight  control,  weapon, 
etc. ) , 


2. 4, 1.4  MIDAS  Limitations — The  format  and  structure  of  MIDAS  is  a  dramatic 
improvement  over  traditional  TO  presentation.  The  MIDAS  job  guides,  for 
example,  are  only  8  in.  long  and  5  in.  wide,  small  enough  for  use  in  more 
confined  areas.  Yet  two  major  difficulties  with  MIDAS  are  still  present. 
First,  there  are  too  many  manuals  and  the  technician  must  cross-reference 
information  from  several  manuals.  Second,  within  a  manual,  obtaining 
information  on  a  specific  aircraft  is  difficult,  requiring  the  technician  to 
identify  the  sections  of  applicable  information  and  filter  or  mentally  "strip 
out”  information  that  is  not  applicable. 

Technicians  in  the  operations  checkout  area  rely  heavily  on  these  manuals,  and 
it  has  been  estimated  that  up  to  501  of  repair  time  is  spent  accessing  and 
locating  information.  Often,  two  technicians  are  required  to  do  a  job  that 
could  be  accomplished  by  one;  the  second  individual  serves  as  an  information 
retriever  and  disseminator  only. 

Once  the  relevant  TO  section  is  accessed,  the  technician  must  sort  the  many 
modifications  made  to  the  F-16.  These  modifications  are  cross-referenced  to 
procedures  and  information  by  aircraft  tail  number.  This  information  search 
turns  out  to  be  an  extremely  time-consuming  and  laborious  task. 
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Figure  7  illustrates  the  process  necessary  to  use  the  manuals  for  checkout  and 
trouble-shooting.  This  flowchart  represents  a  segment  of  the  procedures  for 
checkout  and  trouble-shooting  of  the  F-16  autopilot,  found  in  the  Flight  Control 
System  Job  Guide  Manual  (TO  1F-16A-2-27 JG-00-1 )  and  Fault  Isolation  Manual  (TO 
1F-16A-2-27FI-00-1) . 

The  upper  dotted  box  in  Figure  7  presents  a  series  of  checkout  steps  and 
expected  outcomes  found  in  the  job  guide.  The  procedure  begins  with  the  actions 
"Release  paddle  switch,"  then  "Rotate  test  set  altitude  controls  until  aircraft 
altimeter  indicates  30,000  ft."  If  the  first  result  ( ,fHor  izontal  stabilizers 
down  slightly")  is  not  obtained,  the  technician  is  referred  to  a  paragraph  in 
the  fault  isolation  manual.  The  technician  must  then  obtain  that  manual  and 
follow  the  procedure  there  until  the  fault  is  isolated.  Job  guides  then  must  be 
used  to  actually  repair  the  fault. 

The  left  dotted  box  presents  fault  isolation  procedures  and  recommended  actions 
based  on  their  outcomes.  The  variation  in  procedures  based  on  modifications  to 
the  aircraft  can  be  demonstrated  here.  If  continuity  is  not  obtained  after  the 
first  set  of  instructions,  two  actions  are  presented.  The  index  number  (5  or  8) 
refers  to  a  series  of  tail  numbers  found  in  the  beginning  of  the  manual,  as 
shown  in  Table  2.  The  technician  must  refer  to  this  list  to  determine  which 
procedure  is  appropriate  to  the  aircraft  currently  being  worked  on. 

In  flight  test,  there  is  a  great  need  to  provide  discrete,  individually  callable 
procedures  specific  to  the  aircraft  on  which  the  technician  is  working.  Such 
procedures  would  be  the  result  of  integration  within  and  between  these  manuals 
combined  with  cross-referencing  to  the  lists  of  modif ica tions.  Material  not 
relevant  to  a  given  aircraft  would  be  stripped  out  to  produce  a  set  of 
procedures  specific  to  each  individual  tail  number.  A  diagram  of  a  stripped-out 
procedure  is  presented  in  Figure  8. 
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2. 4.1. 5  F-16  I  MAS  Candidacy — It  is  recommended  that  an  authoring  capability  be 
provided  to  compile  aircraft-specific  procedures  on  an  as-needed  basis.  This 
authoring  system  concept  is  described  in  Section  3. 

The  F-16  repair  and  modification  facility  would  benefit  from  an  electronic 
aiding  system.  F-16-related  activities  meet  the  criteria  of  being  long, 
complicated  tasks  on  a  variety  of  equipment,  requiring  table  look-up, 
information  integration,  high  perceptual  demand,  and  high  criticality  of  correct 
performance.  The  activities  of  checkout  and  fault  localization  during 
modification  could  be  done  much  faster  with  an  IMAS  than  with  paper  manuals. 
Using  the  manuals  as  references  for  solving  problems  not  procedural ized  in  the 
TO  would  also  be  feasible  on  an  IMAS. 

However,  it  is  not  clear  at  this  time  how,  or  how  much,  an  IMAS  capability  would 
be  used  for  modifications  in  the  production  area.  The  procedures  for  aircraft 
modification  are  specific  to  a  TCTO,  have  a  limited  period  of  use,  and  change 
during  the  period  in  which  the  modification  is  being  accomplished.  An  IMAS  is 
not  likely  to  be  cost-effective  for  this  activity. 

2.4.2  Precision  Measurement  Equipment  Laboratory 

PMEL  supports  all  other  maintenance  areas  at  Hill  AFB  by  inspecting, 
calibrating,  repairing,  and  overhauling  all  measurement  equipment  required  for 
aircraft  maintenance.  The  Electro  Mech/Optical  Dimensional  Section  is 
responsible  for  approximately  2000  devices,  such  as  pressure  transducers, 
optical  alignment  devices,  weighing  scales,  and  guided  weapon  test  sets. 
Equipment  is  sent  to  PMEL  either  for  periodic  maintenance  or  when  it  is 
suspected  to  be  faulty. 

PMEL  also  supports  organizational- level  maintenance  activities.  Vhen 
calibration-related  problems  occur  in  organizational  maintenance,  the  equipment 
requiring  calibration  is  either  sent  to  the  PMEL  facility  or  PMEL  technicians 
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travel  to  the  site,  where  they  either  perform  the  calibration  or  determine  if 
the  problem  is  one  of  calibration  at  all* 

2. 4. 2.1  PMEL  Technician  Activities — Significant  PMEL  time  is  spent  writing 
technical  documentation.  When  sufficient  information  is  not  available  for 
organizational-level  technicians  to  perform  calibration  and  other  related  tasks, 
PMEL  technicians  will  write  TOs  for  these  tasks,  to  be  disseminated  at  the 
organizational  level.  In  addition,  PMEL  experts  must  often  write  calibration 
and  maintenance  TOs  if  no  documentation  exists.  They  also  write  supplementary 
material  to  clarify  existing  (usually  commercially  produced)  TOs. 

Calibration  of  a  particular  measurement  device  requires  knowledge  at  several 
levels  of  generality. 

•  The  technician  must  have  knowledge  of  the  theory  of  measurement  for  a 
device  (e.g.,  optics,  electricity). 

•  The  technician  must  be  familiar  with  supporting  equipment  used  for 
calibrating  a  device  (e.g.,  simulators,  frequency  counters)  and  with  its 
application  to  the  particular  device  being  calibrated. 

•  Since  some  of  the  equipment  used  at  Hill  AFB  is  more  than  20  years  old, 
the  technician  must  be  familiar  with  several  models  of  supporting 
equipment  and  the  various  underlying  principles  or  technology. 

Much  of  the  equipment  maintained  at  PMEL  is  seen  only  at  regular  intervals, 
which  may  span  up  to  six  months.  It  is  not  cost-effective,  therefore,  for  PMEL 
technicians  to  maintain  one  particular  device.  Although  these  technicians  are 
certified  in  particular  areas,  they  are  required  to  maintain  a  large  variety  of 
equipment  in  many  disciplines.  Frequently,  intensive  exposure  to  a  particular 
piece  of  equipment  is  not  the  rule,  making  memorization  of  procedures  difficult 
and  close  use  of  TOs  necessary. 
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2. 4. 2. 2  PMEL  Task  Information — The  TOs  used  by  PMEL  technicians  are  much  less 
complete  than  those  in  other  areas*  Many  PMEL  TOs  are  written  on  the  assumption 
that  the  basic  procedures  and  related  knowledge  are  already  known  by  the 
reader.  These  TOs  are  basically  expert- to- expert  communications,  since  many 
steps  are  not  spelled  out  in  enough  detail  for  a  nonexpert  to  follow.  For 
instance,  many  TOs  provide  calibration  procedures  formatted  in  a  step-by-step 
list  where  the  level  of  description  and  supporting  information  is  appropriate 
only  for  the  experienced  technician  who  has  performed  the  task  before  and  needs 
little  elaboration  on  the  basic  procedures. 

For  new  or  inexperienced  PMEL  technicians,  additional  detail  is  sometimes 
unavailable  except  from  other  PMEL  technicians.  For  example,  some  TOs  provide 
information  on  how  to  measure  a  particular  tolerance  value,  but  exclude 
procedures  for  adjusting  the  device  when  it  is  out  of  tolerance.  In  most  cases, 
no  rationale  or  explanation  for  the  procedures  is  provided,  making  the 
procedures  difficult  to  relate  to  what  a  new  technician  may  already  know  about 
the  principles  of  operation  of  a  device. 

When  additional  information  is  available,  it  lacks  integration.  Technicians 
must  spend  time  searching  for,  and  keeping  track  of,  information  scattered 
throughout  one  or  more  TOs.  Too  much  information  is  provided,  however,  for  the 
experienced  technician,  who  must  spend  time  looking  for  specific  data. 

This  situation  has  resulted  in  the  creation  of  subject  matter  experts  at  PMEL. 
These  are  technicians  with  many  years  of  experience,  who  have  learned  the 
procedures  not  available  in  the  TOs  and  are  the  only  individuals  qualified  to 
perform  some  activities.  For  example,  only  two  individuals  can  presently 
calibrate  the  MC-1  compass  calibrator  set,  and  only  four  can  calibrate  the 
Heads-Up  Display  (HUD)  alignment  equipment.  PMEL  is  forced  to  rely  on  a  small 
number  of  experts,  and  their  unavailability  can  cause  delays  in  maintenance 
activities,  both  at  PMEL  and  the  aircraft  maintenance  facility. 


2.4.2 .2  PMEL  I  MAS  Can^^acy — Because  many  tasks  are  performed  relatively 
infrequently  and  on  an  extremely  wide  range  of  equipment,  PMEL  satisfies  many  of 
the  criteria  for  application  of  job  aiding.  Calibration  of  optical  alignment, 
boresighting,  and  electronic  frequencies  are  long,  complex  procedures.  Aiding 
is  needed  for  these  tasks.  Calibration  of  some  devices,  such  as  pressure 
transducers  and  weighing  scales,  is  fairly  standard  and  independent  of  specific 
devices.  They  present  few  problems. 

The  use  of  aiding  includes  support  for  long-term  memory  (e.g.,  procedures), 
short-term  memory  (e.g.,  calculation),  and  perception  (e.g.,  equipment 
subcomponent  identification),  A  method  of  writing  TOs  and  incorporating  aiding 
in  task  performance  is  highly  recommended.  It  would  be  extremely  beneficial  to 
allow  less  experienced  personnel  to  perform  without  expert  intervention  and  to 
preserve  the  PMEL  expertise  that  is  now  handed  off  verbally. 

In  sum,  PMEL  would  benefit  from  an  electronic  aiding  system.  PMEL  activities 
meet  the  criteria  of  being  long,  complicated  tasks  on  a  variety  of  equipment, 
requiring  table  look-up,  information  integration,  high  perceptual  demand,  and 
high  criticality  of  correct  performance. 

2.4.3  Landing  Gear  Division 

The  Landing  Gear  Division  is  an  industrial,  assembly- line  plant  for  the 
inspection,  retooling,  and  assembly  of  landing  gears.  When  landing  gear 
arrives,  it  is  first  completely  disassembled.  The  component  parts  are  inspected 
and  sent  to  the  different  areas  of  the  shop.  Automation  has  been  introduced  in 
several  operations,  especially  those  that  present  potential  risk,  such  as 
removing  paint,  cleaning,  and  impregnating  parts  with  dye  for  subsequent  visual 
inspection  for  structural  defects.  Other  inspections  are  manual  and  visual, 
such  as  checking  diameters  of  pieces  and  holes.  Components  that  do  not  meet 
tolerances  are  discarded  or  sent  to  a  machine  shop  for  reworking.  Finally, 
landing  gears  are  teassembled  using  rebuilt  pieces. 
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Althougn  this  area  was  considered  a  prime  candidate  for  an  IMAS  before  closer 
inspection,  we  have  concluded  that  there  is  little  need  for  an  IMAS  in  this 
facility.  Most  jobs  are  highly  structured,  standardized,  and  repetitive.  Most 
technicians  do  the  same  job  for  periods  of  several  months  and  perform  the  same 
tasks  many  times  a  day.  Technicians  can  soon  remember  the  procedures  by  rote 
and  rarely  need  to  consult  the  TOs.  Therefore,  it  is  difficult  to  see  any  clear 
benefits — like  reduced  numbers  of  personnel,  skill  demands,  or  performance 
time — that  can  be  related  to  significant  increases  in  productivity  or  quality  of 
maintenance. 

However,  there  is  a  need  in  the  Landing  Gear  Division  for  a  planning  and 
tracking  system.  The  primary  need  is  to  compile,  report,  and  update  time  and 
material  standards  for  the  repair  jobs.  This  kind  of  system  is  easily  within 
the  state  of  the  art  in  management  information  systems.  The  principal 
activities  to  be  supported  are  collating,  sorting,  and  reducing  a  large  volume 
of  data  on  job  times  and  material  usage.  The  tracking  system  would  require  an 
interface  with  the  work  flow  to  provide  job  performance  data.  The  planning 
system  would  require  an  interface  with  ATOS  for  applicable  planning  data. 

2.4.4  Sheet  Metal  Shop 

Metal  working  is  not  a  candidate  for  an  IMAS  application.  Most  of  the  work 
involves  modifying  parts.  A  particular  modification  is  done  repetitively  until 
all  aircraft  in  the  inventory  have  been  modified.  The  technicians  soon 
overlearn  the  procedures  and  associated  data  and  can  perform  the  tasks  without 
referring  to  procedural  guidelines.  The  perceptual/cognitive  demands  are 
initially  impressive,  but  the  frequency  of  performance  eliminates  memory 
problems.  The  technicians  seem  to  knew,  for  example,  the  location  and  size  of 
every  rivet  in  the  wing  assembly  on  which  they  are  working.  Some  manual  skills, 
such  as  operating  an  electric  drill,  are  critical,  but  they  cannot  be  aided  by 
an  IMAS. 
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There  was  a  high  training  need  in  this  facility  at  the  time  of  this  study.  This 
need  was  short-term  and  atypical,  however.  It  arose  from  an  expansion  of 
workload  created  by  a  wing  modification  for  one  type  of  aircraft, 

2.4.5  Automatic  Test  Equipment  (ATE)  Facility 

Technicians  in  the  ATE  facility  could  benefit  from  some  kind  of  aiding, 
especially  in  fault  localization  after  the  ATE  has  narrowed  the  source  to  a 
limited  number  of  components.  However,  this  need  is  not  adequately  defined  to 
assess  the  applicability  of  an  IMAS. 

TWo  other  approaches  seem  to  be  more  relevant.  Skill  and  experience  in 
electronics  are  important  and  should  be  given  emphasis  through  training  and 
recruiting  practices.  Unfortunately,  these  qualifications  are  also  in  demand  in 
several  other  jobs  that  provide  greater  financial  rewards.  At  the  ATE  facility, 
reducing  personnel  turnover  may  be  more  useful  than  performance  aiding. 

Further,  integrating  aiding  directly  into  the  ATE  is  a  better  way  to  provide 
support;  no  good  way  of  integrating  an  IMAS  with  existing  ATE  was  apparent, 

2.5  RECOMMENDATIONS  FROM  DEPOT  MAINTENANCE  REVIEW:  AIDING  CANDIDATES 

Several  application  candidates  for  aiding  have  been  identified  and  screened  for 
suitability  as  prototypes  for  demonstration  and  evaluation.  Construction  of 
these  prototypes  is  recommended  as  the  first  phase  in  a  long-range  program. 

Recommended  applications  are 

•  Integration  of  TOs  for  checkout  of  the  F-16  flight  control  system 

•  Calibration  of  equipment  used  to  maintain  the  MC-1  magnetic  compass 

•  Calibration  of  equipment  for  boresighting  the  20-mm  cannon 

•  Calibration  of  equipment  for  optical  alignment  of  the  F-16  HUD 
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These  candidates  have  been  chosen  on  the  basis  of  the  selection  criteria 
described  in  prior  sections.  There  is  a  clear  need  for  aiding  in  these  areas 
among  those  that  were  reviewed  at  Hill  AFB.  There  is  potential  benefit  in  terms 
of  reduced  task  time,  task  difficulty,  and  probability  of  error.  These  factors, 
in  turn,  will  reduce  ability  requirements,  training  time,  and  the  number  of 
people  needed  to  maintain  the  aircraft  systems.  An  IMAS-type  system  also  is 
compatible  with  the  activities  and  work  situation  at  depot. 

This  study  effort  was  restricted  in  scope  and  did  not  include  a  formal, 
quantitative  cost/benefit  analysis.  It  was  intended  as  a  front-end  analysis  of 
feasibility  and  potential  benefit  and  as  a  projection  of  how  to  implement 
interactive  aiding  in  depot  maintenance.  These  recommendations  reflect  our 
best,  experienced,  quantitative  judgment  and  are  based  on  the  information  we 
were  able  to  collect  in  the  time  allotted,  limited  on-site  discussions  with 
maintenance  and  planning  personnel,  and  our  evaluation  of  that  information. 

More  detailed  analysis  of  this  depot-level  problem  depends  on  def inii  a  more 
detailed  IMAS  concept  for  these  applications  and  then  devising  a  work  plan  for 
implementation.  Estimates  of  cost  factors  are  dependent  on  this  level  of 
detail.  Subsequent  sections  of  this  report  will  contain  a  system  concept  for 
interactive  maintenance  aiding,  a  definition  of  two  key  prototypes  to  be 
developed,  and  a  long-range  development  plan  for  implementing  interactive 
maintenance  aiding  at  the  Ogden  ALC. 

The  two  recommended  prototypes  are 

•  An  IMAS  for  the  integration  of  maintenance  instructions  for  checkout  and 
fault  isolation  in  the  F-16  flight  control  system 

•  An  IMAS  for  calibration  of  the  MC-1  compass 
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The  flight  control  system  is  a  complex  but  representative  area  in  F-16 
maintenance.  We  have  reviewed  the  TOs  and  believe  that  adequate  information 
exists  to  proceed  with  design  and  implementation.  The  resulting  system  will  be 
adaptable  to  other  F-16  areas.  The  MC-1  calibration  task  is  one  of  three 
applications  we  have  identified  within  PMEL.  The  MC-1  compass  prototype  is 
recommended  first  because  it  is  smaller  in  size,  adequately  structured,  and 
representative  of  the  needs  and  problems  in  PMEL. 


SECTION  3 


PRELIMINARY  INTERACTIVE  MAINTENANCE- AIDING  SYSTEM  CONCEPT 


In  this  section,  a  preliminary  IMAS  concept  for  application  at  Hill  AFB  is 
described  and  basic  functional  specif ica tions  are  presented.  A  development 
roadmap  for  individual  application  testbeds  and  an  implementation  roadmap  for 
expansion  to  other  tasks  on  the  base  are  discussed. 

3.1  IMAS  CONCEPT 


Performance  aiding  using  electronic  documentation  can  be  very  different  from 
paper-oriented  aiding.  The  translation  of  paper  aids  into  on-line  electronic 
aiding  requires  more  than  reprocessing  words  if  advantage  is  to  be  taken  of  the 
great  potential  of  the  medium. 

Therefore,  electronic  documentation  is  not  just  electronic  page- turning.  It  is 
a  procedure  for  presenting  information,  not  just  the  information  per  se.  There 
are  large  benefits  to  be  gained  by  recording  paper-oriented  information  in 
electronic  form  for  the  purpose  of  updating,  publishing,  and  distribution. 
However,  there  are  forms  of  information  recorded  by  the  computer  that  are  not 
easily  translatable  into  a  paper  equivalent.  To  take  advantage  of  the  full 
range  of  interface  techniques  that  can  be  obtained  with  electronic 
documentation,  the  paper  metaphor  must  be  augmented. 

3.1.1  System  Overview 


The  objective  of  the  IMAS  concept  is  to  provide  the  technician  with  information 
aids  that  will  assist  in  long-term  memory  storage,  short-term  memory  capacity, 
perceptual  limitations,  and/or  reasoning  capability.  The  aiding  techniques 
should  take  advantage  of  the  large  potential  of  the  electronic  medium.  The  two 
basic  components  of  the  proposed  IMAS  are  designed  to  achieve  this  objective. 
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•  IMAS  will  provide  a  library  of  work  packages  to  the  technician,  created 
from  multiple  information  sources,  including  ATOS  files, 

•  An  authoring  system  will  be  provided  to  allow  interactive  creation, 
modification,  and  updating  of  interactive  maintenance-aiding  material, 
and  to  facilitate  the  integration  of  information  from  multiple  sources 
into  work  packages. 

Work  packages  are  sets  of  software  consisting  of  highly  cross-indexed  and  easily 
accessible  procedures  and  data.  They  are  the  final  aiding  product  to  be  used  by 
technicians  on  delivery  hardware. 

The  work  packages  will  be  created  by  the  use  of  an  authoring  system  and 
supplementary  compilation  functions,  as  shown  in  Figure  9.  A  good  authoring 
system  guides  an  author  through  the  writing  process  by  a  series  of  menus  that 
permits  the  author  to  choose  the  information  to  present,  the  format,  and  the 
method  of  presentation. 

Procedural  and  graphical  information  from  the  current  TOs  is  stored  in  digitized 
form  on  ATOS.  ATOS  files  and  other  supplemental  information  (e,g.,  selected 
aircraf t  maintenance  histories)  will  be  entered  into  the  IMAS  authoring  system, 
where  it  will  be  integrated  with  advanced  aiding  techniques  by  a  combination  of: 

•  Automated  creation  of  default  authoring  choices  by  the  system 

•  Handcrafting  of  aiding  techniques  interactively  selected  by  the  author 

•  Automated  integration  functions  (e.g.,  automatic  access  of 
cross-referenced  information  sources  and  documents) 

In  the  recommended  F-16  application,  the  work  packages  will  be  constructed  in  a 
two-step  process  that  will  strip  off  information  not  related  to  the  specific 
aircraft  being  modified.  The  authoring  system  will  allow  the  creation  of 
appropriate  aiding  for  all  segments  of  possible  F-16  maintenance  procedures. 
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.  Architecture  of  the  Major  Components  of  IMAS 


These  procedural  segments  are  then  available  for  compilation  into  final  work 
packages  containing  only  those  procedures  applicable  to  a  particular  aircraft. 

In  this  way,  entry  of  a  particular  aircraft  tail  number  into  the  system  will 
call  up  its  modification  history  from  the  existing  ACMS.  Using  this  history, 
the  system  can  selectively  compile  the  relevant  preauthored  work  package 
segments  containing  those  maintenance  procedures,  schematics,  illustrated  part 
breakdowns,  etc,,  that  are  applicable  to  the  particular  aircraft. 

The  resulting  work  package  data  and  software  will  then  be  available  for 
subsequent  use  by  the  technician  in  on-the-job  delivery  hardware.  Running  this 
software  program  on  delivery  hardware  yields  the  final  system  output:  graphics 
and  text  overlays  combined  to  form  an  integrated  information  access  and  display 
that  is  optimized  for  the  maintenance  technician's  tasks. 

This  same  system  can  be  used  for  any  application  area  where  there  is  more  than 
one  procedure  for  a  particular  type  of  equipment  because  of  multiple 
configurations.  Though  a  significant  amount  of  cross-referencing  still  will  be 
required,  the  recommended  PMEL  application  will  require  less  stripping  than  the 
F-16  application,  and  the  work  package  segments  will  be  closer  to  final  work 
package  form.  In  addition,  the  capability  to  create  and  amplify  calibration 
procedures  will  be  supported. 

3.1.2  The  IMAS  Authoring  System 

The  heart  of  the  IMAS  concept  is  the  semiautomated  design  of  the 
maintenance-aiding  interface  and  the  selection  of  interactive  aiding  techniques 
and  job  information  to  be  included  in  electronic  work  packages.  The  IMAS 
authoring  system  is  conceived  as  a  tool  that  helps  transform  paper-oriented 
electronic  documentation,  primarily  from  ATOS,  into  a  portable  (i.e.,  relatively 
machine-independent)  software  program  for  delivering  advanced,  computer-based 
aiding.  The  characteristics  of  the  authoring  system  are  listed  in  Table  3. 
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TABLE  3.  CHARACTERISTICS  OF  THE  PROPOSED  IMAS  AUTHORING  SYSTEM 


•  Interactive 

•  Menu- driven 

•  Input  includes  ATOS  files,  ACMS  maintenance  histories,  and  user 
expertise 

•  Automated  authoring  functions  wherever  possible 

•  Facil itates cross-ref erencing  and  integration  of  relevant  information 

•  Incorporates  and  enforces  advanced  maintenance  interface  techniques 

•  Generates  aiding  options  for  user  when  possible  (e.g.,  highlighting) 

•  Automatic  or  semiautomatic  stripping  function  (dependent  on 
inputs,  e.g,,  ATOS) 

•  Output  is  portable  software  (relatively  machine- independent ) 


The  author  is  expected  to  be  a  skilled  depot-level  engineer  with  personal 
knowledge  of  maintenance  jobs,  TOs,  and  other  information  sources.  The  author 
should  be  in  a  position  to  improve  aiding  packages  based  on  reported  problems  or 
suggestions  from  reputable  depot  technicians.  The  author  will  be  expected  to 
acquire  some  knowledge  of  the  aiding  techniques  available  in  the  authoring 
system,  preferably  through  a  basic  training  and  decision-aiding  facility 
embedded  in  the  system. 


The  system  will  be  designed  for  ease  of  use.  It  will  be  menu-driven  rather  than 
reliant  on  a  specialized  programming  language  and  its  attendant  requirements  for 
expertise  in  direct  programming. 


The  workload  of  the  IMAS  author  will  be  reduced  wherever  possible  by  the  use  of 
automatic  routines  for  integrating  related  documents  and  generating  standardized 
aiding  techniques*  The  system  will  incorporate  as  many  built-in  functions  for 
simplifying  and  standardizing  the  authoring  task  as  possible.  Given  suitable 
input,  the  system  could  generate  the  bulk  of  the  aiding  and  integration 
functions  automatically  for  validation  by  the  author/user.  Given  embedded  cues 
(e.g.,  control  characters)  in  the  ATOS  files  and  other  information  sources,  the 
IMAS  authoring  system  could  generate  a  first-cut  version  of  the  final  work 
packages,  to  be  "f ine-tuned"  by  the  author.  Significant  time  would  be  saved 
compared  to  handcrafting  the  work  package  from  start  to  finish. 

Aiding  rules,  encoded  in  the  authoring  system,  will  enable  the  enforcement  of 
uniform,  standardized  production  of  the  final  aiding  product  by  creating  a  first 
cut  for  subsequent  author  validation.  These  rules  would  be  triggered  whenever 
there  is  sufficient  information  to  enable  their  use.  For  example,  the  proposed 
IMAS  authoring  system  might  take  ATOS-based  text  and  graphics  as  input  and 
produce  a  work  package  segment  that  allows  TOs  cross-referenced  in  a  particular 
screen  of  text  to  be  accessed  by  means  of  a  window  of  menu  items,  each 
corresponding  to  relevant  TO  information.  The  author  would  then  be  given  the 
option  of  including  this  first-cut  menu  in  the  final  work  package. 

A  file  of  interface  techniques,  such  as  those  in  Table  1,  will  have 
corresponding  authoring  functions  built  into  the  IMAS  authoring  system.  The 
effectiveness  of  various  aiding  techniques  at  this  level  has  not  yet  been 
validated.  Therefore,  the  file  of  techniques  will  be  compiled  on  the  basis  of 
engineering  judgment.  It  will  be  modified  as  its  benefit  is  proven  in 
collateral  development  studies. 

The  IMAS  authoring  system  will  contain  a  number  of  automated  functions  that  are 
relatively  independent  of  author  control.  Information  stripping  is  a  critical 
aspect  of  F-16  maintenance  aiding  since  an  extremely  large  portion  of  the 
technician^  time  is  spent  searching  for  information  and  procedures  appropriate 
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to  a  particular  aircraft.  Both  TO  changes  and  conditional  job  guide  procedures 
create  the  opportunity  for  serious  and  costly  errors. 

3.1.3  ATOS  Compatibility 

ATOS  is  designed  to  contain  a  large  amount  of  indexing  information  that  will  be 
needed  for  input  into  the  IMAS  authoring  system,  particularly  for  indexing 
related  documents  (Reference  10).  The  ATOS  file  structure  is  probably  adequate 
for  the  purpose  of  accessing  TO  files  that  are  cross-r ef erenced  within 
documents.  However,  it  will  be  desirable  for  the  authoring  system  to  automate 
aiding  technique  options  within  TO  procedures.  ATOS  provides  the  needed 
capability  of  accessing  intra-TO  information  by  superficial  document  structure 
(e.g.,  page  number,  paragraph  number,  table  or  figure  number,  etc.),  but  it  does 
not  describe  procedures  by  content  or  form  (e.g.,  branching,  conditionals). 

It  may  be  desirable  to  recast  the  simple  character  strings  in  the  ATOS  files 
into  another  form,  such  as  directed  graphs,  a  form  similar  to  a  decision  tree. 
This  form  would  allow  the  use  of  more  highly  automated  authoring  capabilities  so 
that  default  treatments  of  procedural  authoring  could  be  automatically 
provided.  The  authoring  system  could  then  detect  a  conditional  branch  based  on, 
say,  the  equipment  version,  and  do  a  better  job  in  stripping  off  irrelevant 
procedures.  An  authoring  system  can  produce  a  directed  graph  representation  of 
procedures  as  an  adjunct  to  its  final  aiding  product,  but  a  tradeoff  analysis 
remains  to  be  performed  to  determine  the  most  effective  time  and  way  for  this 
information  to  be  entered. 

To  author  TOs  and  corresponding  work  packages  from  scratch,  the  IMAS  authoring 
system  can  be  used  in  conjunction  with  the  planned  ATOS  authoring  system.  The 
nature  of  this  integration  awaits  further  study. 

The  IMAS  authoring  system  will  be  fully  compatible  with  all  ATOS  files  and 
capable  of  reading  all  special  embedded  control  characters.  The  authoring 


system  will  also  need  to  extract  cues  from  the  ATOS  conventions  in  order  to 
automatically  generate  aiding  options  for  the  author.  Further  study  needs  to  be 
performed  on  this  and  other  aspects  of  the  IMAS-ATOS  communications  interface. 

3.2  MINIMUM  IMAS  SPECIFICATIONS 

The  scope  of  this  study  did  not  allow  the  development  of  detailed  specif ica tions 
for  an  IMAS  system*  However,  some  minimise  requirements  were  developed  based  on 
overlap  in  information  delivery  needs  between  several  potential  application 
areas*  A  detailed  functional  specification  awaits  further  study  of  the  ways 
that  available  information  delivery  methods  can  enhance  the  problem-solving 
methods  employed  by  technicians  in  particular  classes  of  tasks. 

3.2.1  Indexing 

A  rich  indexing  systan  will  be  provided  that  allows  easy  access  to  relevant 
information.  The  technician  will  be  able  to  access  all  relevant  information  by 
specifying  the  system  or  device,  using  highlighted  or  menu  selections  from  the 
current  display,  or  employing  user- transparent  access  and  display  whenever 
possible.  For  information  that  cannot  be  supplied  directly  to  the  interface 
(e.g.,  training  packages,  video  tapes,  maintenance  histories),  the  system  will 
specify  the  method  of  obtaining  that  information  in  a  convenient  and  easily 
understandable  manner. 

Cross- indexing  will  be  achieved  in  a  user- transparent  manner.  When 
cross-indexing  occurs  between  relevant  files,  the  authoring  system  will  enable 
the  prestructuring  of  work  packages  so  that  the  burden  will  fall  on  the  delivery 
system  rather  than  the  technician  to  access  and  display  that  information.  The 
technician  will  use  menu  selection  by  mouse,  touch  screen,  or  cursor  control  to 
access  cross-referenced  information.  An  extended  example  of  this  concept, 
accomplished  by  highlighted  key  words,  is  presented  irr  Figures  2  through  6. 
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3,2.2  Information  Filtering 


When  procedures  vary  according  to  equipment  version,  only  those  procedures  that 
are  applicable  to  the  present  equipment  are  displayed.  This  is  particularly 
applicable  to  tasks  like  F-16  modifications  in  which  each  aircraft  has  had  a 
different  set  of  modifications  performed  on  it,  thereby  making  a  standard  set  of 
procedures  impossible  to  construct.  Present  TOs  have  a  confusing, 
time-consuming,  and  error-prone  system  of  determining  which  procedures  apply  to 
which  aircraft.  An  important  feature  of  an  IMAS  is  the  ability  to  display  only 
those  procedures  applicable  for  a  particular  technician  working  on  a  particular 
aircraft  with  particular  modifications. 

3.2.3  Interface  Display  Techniques 

Information  will  be  displayed  according  to  principles  consistent  with  research 
in  computer-user  dialogue,  attentional  focus,  information  integration,  and  other 
human  factors  that  may  impact  speed  and  accuracy  of  performance.  A  provisional 
list  of  techniques  and  their  uses  is  presented  below,  in  rough  order  of 
applicability.  Tradeoff  analysis  is  necessary  to  determine  the  full 
requirements  for  each  operational  organization  (for  example,  PMEL  or  F-16). 

•  Highlighting:  Illuminating  areas  of  the  screen  to  attract  attention  to 
a  key  event 

•  Hypertext:  Highlighting  key  words  and  elicitation  of  supporting  or 
explanatory  information  by  touch,  mouse,  or  cursor  mechanisms;  a  method 
for  presenting  definitions,  higher  levels  of  explanatory  detail, 
cross-referencing,  or  deeper  menu  structure;  also  used  to  provide  low 
level  of  detail  for  experienced  technicians  while  still  providing 
another  "layer"  of  explanation  for  less  experienced  personnel 
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•  Split  screen  techniques:  The  simultaneous  presentation  of  text, 
graphics,  segments  from  multiple  information  sources,  "folding"  of 
schematics,  etc. 

•  Windowing:  Similar  to  split  screen  techniques;  used  for  temporary 
displays  like  menus  and  warnings 

•  Overlay  techniques:  Methods  for  decluttering  detailed  graphics  and 
filtering  out  unneeded  visual  information;  similar  to  cellophane 
overlays  in  paper  media 

•  Scrolling,  panning,  and  zooming  in  an  area  or  block:  Methods  to  move 
digitized  graphics  up,  down,  right,  and  left,  and  to  change  the 
pictorial  scale  to  a  higher  or  lower  resolution 

3.2.4  Special  Information  Content 


Specialized  integration  aids  will  provide  the  means  to  use  information  from 
traditionally  separate  sources. 

•  Bookmarking:  The  capability  of  ’^marking"  a  place  as  in  a  paper  TO  so 
that  a  technician  can  interrupt  a  task  and  continue  later  without 
backtracking 

•  No-fault  values:  The  option  of  supplying  nominal  values  directly  on  a 
graphic  rather  than  having  technicians  manually  annotate  drawings 

•  Functional  block  diagrams  with  animated  input /output  flow:  A  method  of 
showing  information  process  flow  between  functionally  distinct 
components  of  the  system  under  maintenance 
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•  Functional  block  diagrams  with  varying  detail  of  functionality  on 
request:  Display  of  a  subsystem  or  component  at  the  next  level  of 
detail 

•  Connection  highlighting:  A  method  of  graphical  selection  of  the  input 
or  output  point  for  signal  tracing 

•  User  calculation/automatic  calculation:  A  system  capability  for 
performing  complex  or  lengthy  mathematical  calculations  for  the  user 

•  Annotation:  The  capability  to  make  notes  on  graphics  or  "in  margins" 
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SECTION  4 


IMAS  DEVELOPMENT  PLAN 

4.1  ROADMAP  FOR  IMAS  IMPLEMENTATION  AT  OGDEN  ALC 

A  three-year  roadmap  for  implementing  an  interactive  maintenance-aiding  system 
for  depot  maintenance  and  support  activities  at  Ogden  ALC  is  presented  in  Figure 
10.  The  basic  concept  is  to  develop  aiding  and  authoring  system  prototypes  in 
selected  application  areas  in  order  to  test  key  concepts  on  a  small  scale. 

These  prototypes  will  then  be  extended  to  more  comprehensive  implementation 
after  the  concepts  have  been  validated. 

The  key  to  successful  IMAS  implementation  is  the  development  of  specific 
prototypes,  represented  in  this  roadmap  as  the  target  of  first-phase 
activities.  IMAS  prototype  development  calls  for  two  parallel  projects:  an  F-16 
application  for  checkout  of  the  flight  control  system  and  a  PMEL  application  for 
calibration  of  optical  equipment.  These  projects  will  be  described  in  more 
detail  in  subsequent  sections. 

The  purpose  of  prototype  development  is  to  provide  an  application  testbed  at  the 
depot  level  and  a  near-term  interactive  maintenance-aiding  product  that  then  can 
be  used  in  a  long-range  plan  of  system  development.  The  prototype  can  be  used 
to : 

•  Test  interface  techniques  for  interactive  maintenance  aiding 

•  Gain  experience  with  the  pragmatic  factors  of  implementation  in  the 
operational  environment 

•  Demonstrate  the  utility  of  the  system 

•  Generate  realistic  quantitative  estimates  of  cost/benefit  components 
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In  developing  the  authoring  system,  the  testbed  will  be  particularly  useful  as  a 
vehicle  for  interactive  use  of  available  information  from  ATOS  and  other 
sources,  including  experts. 

In  addition  to  the  central  activity  of  prototype  development,  the  IMAS  roadmap 
includes  a  timely,  comprehensive  survey  of  other  maintenance  and  support 
activities  at  Hill  AFB,  to  be  undertaken  during  the  first  two  phases.  This 
survey  should  be  conducted  as  the  recommended  prototypes  are  being  developed. 

The  results  of  the  survey  should  be  used  to  assess  each  additional  area  for 
potential  benefit  from  aiding  and  to  assign  a  priority  to  each  candidate 
application  on  this  basis. 

It  is  recommended  that  some  collateral  technology  issues  be  addressed  as  part  of 
the  IMAS  roadmap.  Study  will  be  needed  on: 

•  The  specific  task  conditions  under  which  particular  aiding  solutions 
should  be  invoked  (automatically  by  the  authoring  system  where  possible) 

•  The  design  of  necessary  help  functions  and  decision  aids  for  the  author 

•  The  desirability  of  embedded  training  functions  that  could  be 
incorporated  as  part  of  the  interface's  explanation  capabilities 

Other  DoD  programs  will  be  addressing  similar,  related  issues  and  applications. 
Relevant  Air  Force  programs  include  ATOS,  the  Integrated  Maintenance  Information 
System  (IMIS),  and  the  Flight  Control  Maintenance  Diagnostics  System  ( FCMDS) . 

The  IMAS  implementation  roadmap  will  allow  the  identification  of  potential 
problems  common  to  all  of  these  programs  before  large-scale  implementation  is 
attempted. 

There  undoubtedly  will  be  continuing  progress  over  the  course  of  IMAS 
implementation  in  areas  such  as  artificial  intelligence,  intelligent  systems, 
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and  virtual  interfaces  to  computer  systems*  It  is  implicit  in  our  roadmap  that 
these  developments  will  be  monitored  and  incorporated  as  feasible.  However,  it 
is  imprudent  to  predict  and  rely  on  specific  functional  breakthroughs  and  we 
will  not  attempt  to  do  so. 

4.2  PHASES  OF  PROTOTYPE  DEVELOPMENT  AND  IMPLEMENTATION 

Development  of  the  two  prototype  applications  will  follow  the  same  sequence  of 
generic  steps.  IMAS  development  and  implementation  consists  of  four  phases.  It 
is  assumed  that  a  maintenance  application  area  with  payoff  for  information 
aiding  and  a  representative  set  of  tasks  have  been  selected  to  demonstrate  the 
suggested  capabilities  of  the  system. 

•  Phase  1  consists  of  developing  a  detailed  design  and  implementation  plan 
for  the  prototype.  Detailed  task  analyses  of  the  jobs  to  be  aided  and 
information  flow  diagrams  of  the  authoring  and  aiding  functions  will  be 
developed.  The  objectives  of  this  phase  are  to  provide  the  following 
inf  orma tion : 

— Performance  specifications  for  the  system  created  by  analyses  of  the 
maintenance  activity 

— A  system  design  concept  at  the  level  of  system  functions  or  "black 
box"  components 

— A  detailed  engineering  specification  for  fabrication  of  the  aiding 
system 

The  information  generated  during  this  phase  should  be  adequate  to 
support  the  subsequent  writing  of  a  statement  of  work  for  construction 
or  procurement  of  the  system.  The  information  should  also  be  adequate 
for  specifying  the  job  aiding  required,  providing  a  basis  for 
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cost/benefit  analysis,  and  identifying  collateral  studies  for  capturing 
data  on  specific  design  issues  and  for  rapidly  prototyping  system 
f  eatur es, 

•  Phase  2  consists  of  building  and  installing  the  prototypes  in  the 
selected  job  areas.  Technicians  will  be  familiar  with  the  new  system, 
and  the  prototype  will  be  evaluated  using  performance  data  and 
technicians1  subjective  responses.  The  IMAS  concept  will  be  revised  as 
neces  sary . 

•  Phase  3  consists  of  testing  and  evaluation.  A  test  plan  for  evaluating 
the  effectiveness  of  the  prototypes  will  be  developed,  and  an  adequate 
number  of  users  will  complete  standardized  exercises  to  provide  data 
for  evaluation.  Reports  of  system  deficiencies  and  experience 
accumulated  with  the  system  will  be  compiled  and  used  for  redesign. 

•  In  Phase  4,  the  prototypes  will  be  expanded  to  other  maintenance 
activities  within  the  operational  organization.  The  extension  to  other 
activities  should  be  preceded  by  a  survey  of  these  activities  and  an 
analysis  of  the  potential  benefit  from  aiding.  Implementation  can  then 
be  scheduled  based  on  this  analysis. 

The  plan  calls  for  concurrent,  parallel  development  of  the  prototype 
applications  and  the  authoring  system.  As  aiding  techniques  prove  beneficial, 
they  will  be  incorporated  as  authoring  system  capabilities.  Though  authoring 
functions  will  be  incorporated  only  as  needed  within  a  particular  application, 
a  high  level  of  transfer  to  subsequent  applications  is  anticipated.  Therefore, 
later  applications  are  assumed  to  require  little  modification  of  the  authoring 
capabil ity . 

IMAS  software  should  be  relatively  machine- independent .  Though  standard, 
bench-mounted  CRT  and  keyboard  equipment  will  be  used  during  development, 
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ultimately,  configurations  for  specific  work  environments,  such  as  a  portable 
device  for  flight-line  use,  will  be  designed*  Therefore,  the  aiding  software 
must  be  "portable'*  across  different  types  of  display,  input,  and  storage 
devices.  Because  the  interface  and  information  delivery  prototypes  will  be 
developed  without  commitment  to  a  particular  delivery  hardware,  additional 
synergism  can  be  gained  by  taking  advantage  of  parallel  development  within 
other  Air  Force  programs. 

ATOS  will  have  to  be  simulated  for  prototype  development.  Other  related  systems 
and  sources  of  information  under  development  may  also  need  to  be  simulated. 
Candidate  aiding  techniques  may  have  to  be  tested  without  waiting  for  the  final 
system  version  or  actual  operational  equipment. 

4.3  PROPOSALS  FOR  DEVELOPMENT  OF  IMAS  PROTOTYPES 

Development  of  two  prototypes  is  recommended.  The  first  recommended  prototype 
is  an  aiding  system  that  provides  a  single  integrated  maintenance  procedure  for 
the  technician  doing  checkout  and  fault  isolation  on  the  F-16.  It  is 

recommended  that  the  prototype  be  limited  to  a  portion  of  the  flight  c c-n-t -r oi . . 

system.  The  second  prototype  provides  aiding  for  optical  calibration  tasks  at 
PMEL.  The  initial  effort  will  be  calibration  of  the  MC-1  compass.  The 
calibration  prototype  will  then  be  extended  to  boresighting  the  20-mm  cannon 
and  to  the  F-16  HUD. 

4.3.1  F-16  Flight  Control  System  Prototype 

The  proposed  aiding  system  will  display  to  the  technician  procedures  for 
checkout  and  fault  isolation  for  the  F-16  flight  control  system.  The  prototype 
will  be  developed  using  a  bench-mounted  CRT  and  keyboard. 

Desired  procedures  will  be  called  by  entering  the  aircraft  tail  number  and  the 
planned  maintenance  action.  Only  the  procedures  specific  to  that  particular 
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aircraft  will  be  presented;  all  other  material  will  be  removed.  Branching 
within  the  procedure  will  occur  on  entry  of  the  outcome  of  a  maintenance 
action.  All  supporting  and  collateral  information,  graphics,  illustrative 
material,  and  aiding  techniques  associated  with  the  presented  information  will 
be  incorporated  as  an  integral  part  of  the  procedures. 

Advanced  aiding  techniques  will  be  incorporated  to  enhance  the  clarity  and 
under standabil ity  of  the  procedures,  reduce  the  cognitive/perceptual  demands 
and  difficulty  of  the  tasks,  and  assist  the  technician  in  tracking  and  managing 
maintenance  activities. 

An  explanation  capability  will  be  provided  to  enable  the  technician  to  request 
additional  information  on  any  task  or  step.  As  examples,  the  explanations  may 
include  diagrams  of  components  at  various  levels  of  detail,  rationale  for  a 
procedure  or  action,  definitions  of  terminology,  tracing  of  electrical  current 
flow,  and  other  material  as  it  is  discovered  to  be  appropriate  to  the 
technician's  tasks  and  ability.  At  least  two  levels  of  description  of  the 
procedures  will  be  provided. 

An  authoring  system  will  be  provided  to  edit  and  compile  ATOS  and  ACMS 
information  and  all  desired  aiding  techniques  into  work  packages  to  be 
delivered  to  the  technician  through  appropriate  delivery  hardware.  The 
authoring  system  will  assist  the  author  in  the  incorporation  of 
aircraft-relevant  material,  cross-referenced  ATOS  files,  and  aiding  techniques 
into  work  package  segments.  These  segments  will  be  available  for  compilation 
by  supplementary  program  functions  into  final  work  packages. 

The  aiding  interface  used  by  the  technician  should  be  eventually  incorporated 
into  a  lightweight,  portable,  hand-held  device  such  as  the  Grid  Compass 
computer  being  developed  under  AFHRL1 s  IMIS  program.  The  portable  device 
should  be  explosion-proof  for  use  on  the  flight  line.  The  display  must  be 
legible  in  the  range  of  ambient  illumination  encountered  in  depot  and 
organization  maintenance  areas. 
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Technicians,  supervisors,  and  planners  will  be  consulted  during  the  development 
of  performance  specifications,  design,  and  implementation*  User  reaction  and 
evaluation  of  projected  ease  and  usefulness  of  system  features  will  be 
obtained.  Consistency  between  user  evaluation  and  performance  data  obtained 
from  collateral  studies  will  be  sought. 

A  roadmap  of  activities  in  developing,  testing,  and  expanding  the  F-16 
prototype  is  presented  in  Figure  11.  Each  of  the  phases — development  of 
requirements  and  design,  build,  and  test  and  evaluation — are  allocated  one  year 
for  completion. 

4.3.2  PMEL  Calibration  Task  Prototype 


Three  tasks  have  been  chosen  for  initial  prototyping: 

•  Calibration  of  the  MC-1  and  MC-1M  compass 

•  Calibration  of  boresighting  the  20-mm  cannon 

•  Calibration  of  the  HUD  alignment  equipment 

These  tasks  were  chosen  because  the  procedures  are  very  detailed,  are  performed 
infrequently,  satisfy  the  criteria  for  aidability,  and  have  available  TOs.  The 
compass  calibration  is  the  simplest  of  the  three  candidates.  Designing  aiding 
and  authoring  systems  for  this  task  will  provide  a  baseline  for  expansion  in 
the  other  two  areas. 

The  MC-1  and  MC-1M  compass  calibration  equipment  provides  magnetic  inputs  to  a 
magnetic  compass  system.  The  differences  between  the  MC-1  and  MC-1M  are  in  the 
control  console,  power  supply,  adapter  box,  slaving  control,  and  interconnector 
cables.  The  frequency,  dc  voltage,  regulation,  and  synchro  control  of  this 
equipment  are  calibrated  by  PMEL  personnel.  Procedures  for  both  compass  sets 
are  contained  in  one  set  of  TOs  (TO  33K6-4-94-1  for  calibration  and  TO 
5N3-3-7-3  for  overhaul). 
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Figure  11.  Roadmap  for  Developing  a  Prototype  IMAS  for  P-16  Checkout 


The  HUD  alignment  set  is  used  for  calibrating  the  Heads-Up  Display  on  the 
F-16.  This  equipment  must  be  calibrated  every  six  months,  and  the  process 
takes  roughly  four  to  six  weeks. 

Table  4  is  a  list  of  the  capabilities  of  the  PMEL  prototype  sytem.  Because 
most  calibration  procedures  are  done  in  the  facility,  benchtop  CRT  terminals 
would  be  the  logical  primary  interface  for  the  technician.  Some  calibration 
procedures  must  be  performed  outside  of  the  facility,  so  portable  terminals  may 
also  be  needed. 


TABLE  4.  PMEL  PROTOTYPE  CAPABILITIES 


Capability 

Design  Specification 

Benef its 

Easy  access  to  information  in  TOs 

Indexing 

Less  time  spent  finding 

TOs 

Access  by  menu  selection 

•  Across  related  TOs 

Less  time  spent  searching 

Information  tailored  to  skill 

•  By  key  words  (equip- 

through  TO  for  relevant 

level  of  technician 

meat  numbers,  com¬ 
ponents) 

•  By  major  headings 

•  With  other  information 
sources  (e.g. ,  train¬ 
ing  materials) 

Menu  selection  by 
touch  input 

information 

Integration  of  information 

Split  screen/windowing 

Simultaneous  viewing  of 

capability 

related  information 
(e.g.,  illustrations 
and  descriptions) 

Preparation  of  TO  materials 

Authoring  language 

Less  time  spent  prepar¬ 
ing  TOs 

Preparation  of  training  materials 

•  Provides  ATOS  format 

Less  reliance  on  subject 

•  Automatic  indexing 

matter  experts  at  depot 

with  reLated 
information 

•  Graphics  input 

•  Digitized  photographs 

and  organizational  level 

Keyboard  use  would  be  limited  to  setting  up  the  work  space  for  the  particular 
task  (e.g.,  loading  required  information).  Once  the  information  is  on-line, 
the  technician  could  input  menu  selections  by  touch  to  minimize  the  manual 
interaction  of  the  technician  with  the  terminal.  Maintenance  instructions  will 
be  presented  in  a  menu  format  using  windowing  techniques,  similar  to  the 
example  presented  earlier. 

Occasionally,  calibration  tasks  require  the  technician  to  perform  lengthy 
and/or  complex  calculations  on  measurement  data,  for  example,  calculating  the 
mean  and  standard  deviation  of  several  instrument  readings.  Current  TO  format 
provides  a  worksheet  for  these  procedures.  These  worksheets  have  a  space  for 
user  data  inputs  and  provide  instructions  for  required  mathematical 
manipulations.  The  IMAS  system  should  allow  inputs  from  the  keyboard  for  these 
procedures,  perform  the  necessary  calculations,  and  provide  explanatory 
materials  if  desired. 

The  authoring  system  component  of  the  proposed  IMAS  is  a  crucial  part  of  this 
application.  Though  TOs  already  exist  for  the  procedures  selected  for 
prototyping,  the  IMAS  authoring  capability  could  be  readily  tested  by  having 
technicians  enter  supplementary  information  on  existing  procedures  that  are  not 
adequately  detailed  for  nonexperts  performing  the  calibration  tasks.  Once  the 
procedures  have  been  supplemented  for  aiding,  the  prototypes  could  be  evaluated 
for  effectiveness  by  testing  whether  technicians  unfamiliar  with  the  procedures 
could  perform  the  calibration  tasks  more  readily  and  without  consultation  with 
resident  experts. 

A  scheaule  of  activities  for  development  of  the  PMEL  prototype  is  presented  in 
Figure  12. 

4.4  COLLATERAL  ISSUES 

There  are  several  areas  in  which  further  collateral  study  will  be  highly 
beneficial  to  the  design  and  development  of  an  IMAS: 
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REQUIREMENTS  STUDY  FOR 
PROTOTYPE  MC  I 


Roadmap  for  Developing  an  IMAS  for  PME1. 


•  Identification  and  design  of  necessary  help  functions  and  decision  aids 
for  the  author 

•  Determination  of  aiding  techniques  required  by  specific  maintenance 
task  characteristics 

•  Desirability  of  incorporating  embedded  training  function?  into  the 
interfaced  explanation  capabilities 

4.4.1  Help  Functions  and  Decision  Aids 

The  purpose  of  the  authoring  system  component  of  IMAS  is  to  generate  effective 
work  packages  for  use  by  the  technician.  An  effective  authoring  system  must 
provide  a  high  level  of  support  and  decision  aiding  to  the  author.  It  can 
reduce  the  user's  workload  and  enforce  a  reliable  standard  of  quality  in  the 
work  packages  created.  However,  a  system  that  permits  an  author  to  insert 
aiding  indi scr imina tely  can  result  in  work  packages  that  reduce  productivity 
and  encumber,  not  aid,  the  technician.  To  the  extent  that  an  authoring  system 
can  guide  an  author  to  the  correct  choice  of  aiding,  the  product  will  be 
consistently  high  in  quality. 

Therefore,  an  important  objective  is  to  construct  a  clever  system  that  can 
generate  desirable  aiding  whenever  possible,  providing  the  author  with  good 
default  choices.  In  addition,  the  author  should  be  provided  with  an  embedded 
decision  aid  for  selecting  aiding  techniques  based  on  target  task 
characteristics  that  may  not  be  directly  detectable  by  the  system. 

4.4.2  Aiding  Based  on  Task  Characteristics 

Default  aiding  to  the  author  can  be  provided  by  rules  for  the  application  of 
particular  aiding  techniques  contingent  on  task  characteristics  (e.g.,  multiple 
procedural,  benches).  Th«  oKjective  is  to  identify  attributes  of  maintenance 
tasks  that  suggest  particular  forms  of  aiding.  These  attributes  should  be 
systematized  into  a  decision  aid  for  the  author,  providing  automatically 
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generated  aiding  options  where  possible  or  providing  a  help  facility  for 
determining  aiding  techniques  on  the  basis  of  task  characteristics. 

4.4.3  Embedded  Training 

Another  issue  for  study  is  the  determination  of  the  desired  amount  of  embedded 
training  to  be  incorporated  into  the  IMAS  interface.  Menu-driven  explanation 
can  be  used  for  on-the-job  training.  A  set  of  explanations  can  be  designed  to 
lead  a  technician  through  a  complex  task  or  cognitive  procedure.  Diagnostic 
feedback  could  also  be  provided  for  correcting  errors.  The  extent  to  which 
explanation  on  the  job  can  replace  prior  classroom  training  is  an  important 
issue  in  the  utility  of  interactive  aiding  systems.  The  reduction  of  ability 
requirements  and  training  time  may  produce  significant  cost  savings  while 
maintaining  or  enhancing  productivity. 

4.5  WHAT  TO  DO  NEXT 

The  next  action  to  be  taken  is  to  initiate  the  requirements  analysis  for 
prototype  development.  Perhaps  the  quickest  approach  is  to  extend  the  present 
work  with  a  contract  to  develop  the  requirements.  The  level  of  effort  should 
be  one  person-year  executed  in  a  six-month  period.  The  objective  of  the 
contract  should  be  to  provide  the  information  necessary  for  a  statement  of  work 
to  be  used  in  a  procurement  to  build  the  prototype  aiding  demonstration.  The 
study  should  cover  both  the  F-16  flight  control  and  PMEL  applications. 

The  principal  activity  of  the  contract  would  be  a  detailed  analysis  of  the 
tasks  to  be  aided  and  an  information  flow  diagram  of  the  authoring  and  aiding 
functions.  The  objectives  of  the  study  would  be  to  provide  the  following 
information: 

•  Performance  specifications  for  the  system  derived  from  an  analysis  of 
the  maintenance  activity 
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•  A  system  design  concept  at  the  level  of  system  functions  or  "black  box" 
components 

•  A  detailed  engineering  specification  for  fabrication  of  the  aiding 
sy  st  ern 

The  level  of  detail  in  this  information  should  be  adequate  to  support  the 
subsequent  writing  of  a  statement  of  work  for  construction  or  procurement  of  a 
system.  It  should  be  adequate  for  specifying  the  job  aiding  required, 
providing  a  basis  for  cost/benefit  analysis,  estimating  the  levels  of  effort 
required,  and  identifying  collateral  studies  to  capture  data  on  specific  design 
issues  and  to  rapidly  prototype  system  features. 

If  the  requirements  analysis  is  done  on  contract,  the  final  result  of  the  study 
should  be  a  sequence  of  tasks  for  building  the  prototypes.  These  tasks  would 
then  be  converted  to  the  statement  of  work  for  a  request  for  procurement. 
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